Protocol for a multicentre, multistage, prospective study in China using system-based approaches for consistent improvement in surgical safety.

INTRODUCTION: Surgical safety has emerged as a crucial global health issue in the past two decades. Although several safety-enhancing tools are available, the pace of large-scale improvement remains slow, especially in developing countries such as China. The present project (Modern Surgery and Anesthesia Safety Management System Construction and Promotion) aims to develop and validate system-based integrated approaches for reducing perioperative deaths and complications using a multicentre, multistage design. METHODS AND ANALYSIS: The project involves collection of clinical and outcome information for 1 20 000 surgical inpatients at four regionally representative academic/teaching general hospitals in China during three sequential stages: preparation and development, effectiveness validation and improvement of implementation for promotion. These big data will provide the evidence base for the formulation, validation and improvement processes of a system-based stratified safety intervention package covering the entire surgical pathway. Attention will be directed to managing inherent patient risks and regulating medical safety behaviour. Information technology will facilitate data collection and intervention implementation, provide supervision mechanisms and guarantee transfer of key patient safety messages between departments and personnel. Changes in rates of deaths, surgical complications during hospitalisation, length of stay, system adoption and implementation rates will be analysed to evaluate effectiveness and efficiency. ETHICS AND DISSEMINATION: This study was approved by the institutional review boards of Peking Union Medical College Hospital, First Hospital of China Medical University, Qinghai Provincial People's Hospital, Xiangya Hospital Central South University and the Institute of Basic Medical Sciences, Chinese Academy of Medical Sciences. Study findings will be disseminated via peer-reviewed journals, conference presentations and patent papers. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

A multicentre prospective study, with three parallel study stages that naturally balances seasonal variation and backgrounds of patients and medical staff.

Well-developed population strategies in prevention medicine are applied in the relatively new discipline of patient safety by introducing a novel concept of ‘risk population’ into surgical setting.

First to formulate an integrated system for surgical safety that enables a dynamic process of safety study, intervention and refinement.

Lack of ergonomic analysis in the process of developing interventions.

Unable to perform post-hospitalisation follow-ups of surgical complications due to the very large study scale.

Introduction

Background

With an estimated 1 million perioperative deaths globally each year,1 surgical safety has emerged as a crucial health issue in the past two decades, requiring increased financial and research investment.2–4 In developed countries, mechanisms such as outcome reporting have been established to trigger improvements in safety for surgical patients,5 6 and tools (eg, safety checklists) are being enriched with the rapid development in the new discipline of patient safety.7 8 However, despite sporadic successes, the pace of large-scale improvement in surgical safety is slow.9–11 As stressed in the Patient Safety Global Action Summit, held in London in 2016, further research and measurements for patient safety are needed in every country, and more holistic, system-based approaches are highly desirable for the successful implementation of safety interventions.12

Developing countries are home to more than 80% of the world’s population and present an important determinant for achieving widespread improvement in surgical safety.13 For example, in China, the safety status of about 44 million inpatient surgeries performed each year is largely unclear,14 and systematic endeavours to improve surgical quality are rare.15 Given the urgent need for improvement, the Chinese government launched a national project entitled Modern Surgery and Anesthesia Safety Management System Construction and Promotion (MSCP) in 2014.16 This project is based on three new strategies for improvements in methodology. First, we intend to incorporate safety outcome reporting with safety intervention mechanisms to allow intervention schemes to be derived from and validated using real data. Second, the Surgical Patient Safety System checklist17 expanded the WHO surgical safety checklist7 from the operation room to cover the whole clinical pathway. However, the extensive implementation labour required hindered its adoption,18 especially in population intensive settings. Therefore, we plan to develop a new whole pathway interventional system that is delivered in a tailored manner using population-based stratification strategies. Third, we deem inherent patient risk factors as more important than harms induced from medical errors;19 therefore, in addition to the traditional focus on reducing medical staff errors, managing patient risk becomes another more important perspective in carrying out safety interventions. Herein we present a detailed report of the overall protocol of this comprehensive project for surgical safety improvement.

Objective

The objectives of the MSCP project are to:

Develop a management network that integrates surgical outcome reporting, clinical practice evaluation and seamless perioperative safety interventions.

Establish and promote this system as a long-lasting mechanism to reduce perioperative deaths and preventable complications.

Cultivate and elevate surgical safety culture and safety capability among medical staff members.

Main hypotheses

From a population point of view, before undergoing an operation, every surgical patient has a risk of developing surgical complications during or after the operation; therefore, as a whole, they are a ‘risk population’ for surgical complications, which is a unique definition of ‘population’ in the clinical setting. Each individual in this risk population has different personal risk factors (eg, underlying diseases). This means that their corresponding levels of risk for developing surgical complications are not the same, varying from 0 to 1. Under such an assumption, big data accumulated in safety outcome reporting systems can be used to preoperatively stratify the risk population, using representative variables that are predictive of future safety outcomes. These reporting system data can also be used to identify patterns in the occurrence of complications, which can be used to further stratify the complications for different prevention groups among the patient subgroups. Applying this population strategy to the clinical prevention of complications has potential to make safety interventions more targeted and cost-effective. Combining outcome reporting and intervention mechanisms may further evaluate the effectiveness and efficiency of safety interventions.

Methods and analysis

Study design and timeline

The main study in this project uses a multicentre, multistage, prospective design. The timeline of the project is presented in figure 1.

Figure 1

Brief outline of the Modern Surgery and Anesthesia Safety Management System Construction and Promotion (MSCP) project timeline.

More specifically, the three stages of MSCP are

July 2014 to December 2014: Overall design and pilot study.

January 2015 to June 2015: Collection of baseline data for analysis of patterns of complication occurrence and risk factors (Data set 1).

July 2015 to December 2015: Initial formulation of interventions (by expert consensus, directed by risk patterns from data analysis). Evaluation of a readapted WHO surgical safety checklist for the operation room (substudy 1) is also planned during this period.

January 2016 to June 2016: Implementation of the interventions and collection of post-intervention data for the evaluation of effectiveness (Data set 2).

July 2016 to December 2016: Refinement and simplification of the interventions (by empirical evidence from data analysis) and development of strategies to enhance clinical engagement and supervision mechanisms for more effective and efficient implementation.

January 2017 to June 2017: Data collection for the efficiency assessment of the refined system (Data set 3). Interventions for some specific safety issues (eg, emergency surgery as substudy 5) and evaluation of their effectiveness are also planned during this period.

July 2017 to December 2017: summary and promotion.

Planned substudies

The project includes five planned substudies:

Substudy 1: Relaunch and implementation of an operating room surgical safety checklist;

Substudy 2: Development and validation of a uniform index set for surgical complication reporting;

Substudy 3: Validity and improvement of an adverse event reporting system in China;

Substudy 4: Intraoperative body temperature monitoring and outcomes of surgical patients; and

Substudy 5:Pattern of emergent operations and mode for emergency planning.

Study setting

Four academic/teaching general hospitals (all found before 1930s) that are representative of the highest quality healthcare in major Chinese administrative regions were chosen as study centres (figure 2). The selection is also representative of China’s geographic, economic and cultural diversity to consider their impacts on the complication occurrence patterns and effectiveness of safety interventions and the necessity for localised methods. In addition, one cardiovascular specialty hospital (the National Cardiovascular Disease Center) was used as source of specialty guidance.

Figure 2

Geographic distribution of the study sites in the Modern Surgery and Anesthesia Safety Management System Construction and Promotion project. CAMS, Chinese Academy of Medical Sciences.

Organisational structure

An upside down organisational structure20 was proposed for the MSCP, with three project teams (figure 3). At the base is the Central Project Group (CPG) responsible for designing and supervising the entire process, calling for interim seminars and project summit conferences and finalising and disseminating all project materials and protocols. As an intermediate layer, Site Administration Teams take responsibility for the effective performance of the project and authenticity of the results at each site and coordinate interactions among sites and executive units. The top layer comprises Functional Teams, further divided into two branches with: (1) Executive Unit heads as executive directors for data and audits, frontline medical staff members as information providers and policy executives and specialised teams or personnel to enter data, and (2) Supporting Teams responsible for information system construction, maintenance and updates and generation of necessary information (eg, surgery name encoding).

Figure 3

Organisational structure of the Modern Surgery and Anesthesia Safety Management System Construction and Promotion project.

Study subjects

The study sample includes all inpatients aged ≥14 years (no age limit for rachioscoliosis) with a minimum hospital stay of 24 hours, who are undergoing surgery and who require the presence of an anaesthesiologist. Patient enrolment spans three data collection periods: January–June 2015, January–June 2016 and January–June 2017. All surgeons and ward nurses from the surgical departments (including General, Thoracic, Cardio, Vascular, Neurosurgery, Orthopaedics, Urology, Obstetrics, Gynaecology and Otolaryngology departments) and intensive care units (ICU), anaesthesiologists and operation room nurses at the four study sites are required to participate in collecting patient data and conducting safety interventions (more than 4000 medical staff members in total). Some staff members also participate in suggestion collections for safety improvement, attitude surveys, knowledge tests and safety quality checks, as determined by the sample size requirements of the relevant substudies.

Outcome measures and data collection

The major study outcomes are deaths (including inpatient deaths and confirmed deaths within 30 days after discharge), surgical complications during hospitalisation and length of stay. The number of reported complication items is initially planned to be extensive (grouped by items for use in all surgeries (common complications) and items for use in different surgical specialties (specialty-specific complications), collected from experts and published literature) and will be narrowed down in the Stage 3. This process is planned in a substudy on developing a classification of surgical complications for surgical safety reporting, which mainly focuses on the perspective of prevention in addition to the severity of the complications in other available classification systems.21 Results of this substudy will be used for reporting of the overall results of the project. In addition, explanatory variables will be recorded in Stage 1 for the formulation of intervention scheme, including: (1) preoperative patient information (demographics, underlying diseases and physical and laboratory examination results); (2) intraoperative medical information (including surgical complexity, anaesthetic method, antibiotic use, bleeding/transfusion, temperature, staff seniority and interim changes); and (3) postoperative nursing information (use of ICU and special care), which will also be scaled down in the second and third stages for simplified execution.

A uniform electronic data capture (EDC) system will be constructed to facilitate data collection, with four professional subsystems for each executive unit. Information in the wards is directly entered by charge doctors and nurses; only common complications will need to be entered into the ICU system; data in the anaesthesiology and operation room systems are entered by an established data entry team after paper forms are completed by frontline staff. In terms of data security, authority for raw data scanning is restricted to the local department level and only specified project personnel will have access to the entire database.

Quality assurance of data

As a primary data audit, completed data may only be submitted with the electronic signatures of the executive unit heads. Double entry is used for stricter control in the ward and ICU (ie, only complete consistency can lead to submission; otherwise, reminders will pop up highlighting mismatches). A statistical team monitors trends in the incidence of complications in each department twice each month, and a clinical specialist group reviews the accuracy of susceptible low or high rates (compared among departments and hospitals) against the original medical records and reports back corrigenda to the source wards. A data verification team (third party) performs random spot checks of data from the anaesthesiology and operation room systems against logic and the original paper forms every 2–3 months. An item error rate of ≥6.0% will result in rejection and repeated entry. A home page bulletin board on the system updates guidelines, such as the recording roles for patient transfers, and any shared mistakes detected by the CPG. Individual questions and errors can be resolved via a hotline and chat group.

Development of evidence-based interventions

Methods for preventing various surgical complications are generated by expert consensus in a predefined process involving: (1) widespread suggestion collection (from literature reviews and experiences from surgical and nursing experts), initial screening and summarisation; (2) panel discussion and iterative scoring and revision (until rules of consensus are met); and (3) validation in practice and refinement with consideration of broad feasibility. The first two steps (designed in Stage 2) are similar to the Delphi method,22 with an exception that the process is not anonymous as it is too time-consuming in this case and is suggested not always necessary for medical research.23 The last step will be taken in Stage 3.

Using stratification strategies that stem from population studies to integrate such methods, surgery departments will be divided into different intervention groups according to their incidence of complications (Data set 1), after which, three levels of prevention will be hierarchically delivered to different patient groups (figure 4). First, all patients of all surgical categories will receive Generic Prevention, a set of approaches focusing on prophylaxis of infections (prevalent in all surgeries) and basic care standardisation. Second, for Specific Prevention, risks for some selected complications (eg, thrombosis) will be assessed in surgical categories with moderate to high complication incidences, and to patients with high risks, pertinent prevention measures are delivered. Third, to prevent life-threatening complications (eg, stroke, cardiac–cerebral vascular even, respiratory failure, etc), patients undergoing surgical categories of high complication incidences will be further classified into smaller risk groups according to features that are highly suggestive of surgical outcome (eg, physical status score, grade of surgical complexity), and different levels of Intensive Intervention will then be accordingly implemented: (1) timely diagnosis and treatment of preoperative system diseases, (2) special tests and treatments that are highly resource-demanding, and (3) internal medicine consultation.

Figure 4

Stratified strategy for delivering prevention approaches of surgical complications to different groups of patients. (A) all patients are covered in the Generic Prevention; (B) patients with high risks for some selected complications are intended for the Specific Prevention; (C) patients undergoing surgeries of high complication incidences are targeted for the prevention of serious complications and further classified into C1 (low-risk patient), C2 (moderate-risk patient) and C3 (high-risk patient) for different levels of the Intensive Prevention.

System construction

The EDC system for collection of clinical and outcome data will be merged into existing hospital information system (HIS) after reducing the clinical items to report in Stage 3 (only important risk factors for patient stratification that would not be otherwise obtained elsewhere are kept) and the number of complications items in Stage 3 (those with quite rare incidences and are hardly preventable are eliminated). Similarly, the safety intervention system, functioning as a reminder and recorder of the intervention items, is initially designed as a stand-alone system for the purpose of effectiveness evaluation in Stage 2, and will be further scaled down and integrated into the HIS in Stage 3 for testing its durability. To reduce manual workload in entering data, the final system automatically extracts available information (eg, age and body mass index) from the existing HIS via specially designed interface.

Safety key message transformation

An established risk in patient safety is failure to transfer the most important messages concerning patients,24 such as physical condition, type of surgery and other known risks, especially in a complicated surgical setting. To block potential safety leakages between different departments, occupations and personnel, the computerised safety intervention system will also display and remind the staff of these key messages of surgical safety.

Strategies for clinical engagement

An anticipated barrier to the successful implementation of the interventions was inadequate buy-in among frontline medical staff, which is a widely attributed reason for fruitless quality improvement initiatives, even for failure to replicate successes with well-evidenced safety measures.25 26 Accordingly, various approaches are adopted to enhance clinical engagement, among which three major strategies are described below:

Led by the CPG, unit-by-unit education and training sessions are performed before the start of the three stages and data collections to create a better understanding of the project aims, essentials of the strategies and potential impacts on quality of care;

In the first two stages, all the outcome items and intervention methods are proposed, discussed, modified and finalised among frontline medical staff members, so are the ways for implementation and the scheme of task allocation. In this way, higher sense of ownership are built and the interventions can be more adapted to the clinical context (see one report from the project team on the importance of valuing clinical motivation in adapting WHO surgical safety checklist);27

In the third stage, we plan to use a supporting system called ‘Graduated Confirmed Awareness’ as is described in figure 5. With this system, information such as execution of the interventions and outcomes of patients are passed on to superior levels step by step, so that clearer senses of roles can be formed,25 while the staff members remain exempt from direct blame as only anonymous (summarised statistics rather than personal) information are shown to administrators. Another role of the system is the transparent feedback of the rates to all surgical departments, with which the simultaneous association (intervention delivery with patient outcome),28 longitudinal comparison (historical curves) and external contrast (compared with other departments and hospitals) can serve as great driving forces for consistent devotion to safety improvement.

Figure 5

Workflow of a Graduated Confirmed Awareness system to increase fidelity to surgical safety improvement.

Sample size estimation

We used perioperative mortality as the main index to calculate the required sample size for the first two stages to validate the effectiveness of safety interventions. As perioperative death is rare in these four high-quality hospitals (0.60% as estimated using previous hospital registry data), to obtain a 20% reduction in mortality (namely 0.45%) after implementation of the interventions, we needed 36 435 patients for each stage, with a type I error rate of 0.05 and a power of 0.8 to detect such a difference. On the basis of the available history data in the study hospitals, the CPG decided that all eligible patients would be enrolled, as there was expected to be about 40 000 patients each half year, which is just slightly higher than the number needed. Another major reason for this decision is that the patterns of complication incidences may not be homogenous in different hospitals given geographic, economic and cultural diversities; therefore, all eligible patients should be non-selectively enrolled to obtain the distinguishable spectra of surgical complications in each hospital. Similarly, an additional 40 000 surgical inpatients are needed in the third data collection period to test the system’s efficiency and vitality. Therefore, the MSCP project will include 1 20 000 surgical patients in total.

Data analysis plan

To improve the translation and dissemination of the findings in this project into real clinical practice,29 30 the final study findings will be analysed and reported according to the ‘RE-AIM’ standards proposed by Glasgow et al.30 The RE-AIM indices for this study are defined as: (1) Reach, ‘the coverage rate of the surgical inpatients with the study inclusion criteria’; (2) Effectiveness, ‘reduced crude and adjusted mortality, overall and specialty-specific complication rates, and length of stay’; (3) Adoption, ‘proportion of the patients who received at least one intervention’; (4) Implementation, ‘item-specific completion rates’ and (5) Maintenance, ‘month-specific trend curves’. Mann-Whitney U tests (for continuous variables such as age and body mass index) and χ2 tests (for categorical variables such as disease history and surgery type) will be performed to test comparability between patients in the three stages. ORs will be calculated with different levels and combinations of explanatory variables to identify risk factors and practical patterns for patient stratification. Regression models such as logistic regression (for death and single complication analysis) and zero-inflated negative binomial regression (for number of complications, considering variance overdispersion due to zero counts and correlations) will be used to estimate the effect of interventions, adjusted for potential confounders. Completion rates will be compared in the last two stages with χ2 or Fisher's exact tests. All tests will be two-tailed with a 0.05 threshold for statistical significance, and 95% CIs will be calculated for the crude and adjusted rates of complications, ORs and reduction.

Study status

The first two stages have been completed. A third round of data collection started on 1 January 2017 and is anticipated to be completed by 30 June 2017. Meanwhile, approaches to enhance implementation quality and efficacy are under development and refinement. Stage 3 will be finalised at the end of 2017, and paper reporting the main results of the whole project will be submitted for publication. A paper reporting substudy 1 (re-adaptation of WHO surgical safety checklist) has been published.27 Substudies 2–5 are being prepared.

Ethics and dissemination

Research ethics approval

This study was approved by the institutional review boards of Peking Union Medical College Hospital, First Hospital of China Medical University, Qinghai Provincial People’s Hospital, Xiangya Hospital Central South University and the Institute of Basic Medical Sciences, Chinese Academy of Medical Sciences. Each board waived the need for patient and medical staff consent as individual information is not disclosed.

Publication and dissemination

The results from the main study regarding the primary objectives and findings from the substudies will be submitted to peer-reviewed journals, as well as presented at national and international conferences. The technical details of the system construction will be published as patent papers.

Discussion

Patient safety is an immature discipline compared with other fields of medical study.31 The difficulties in its development are based on several aspects. First, assessing safety is more complicated than assessing the effectiveness of a new drug or accuracy of a novel diagnostic device as safety outcomes (surgical complications and other adverse events) are diverse and cannot be uniformly measured, and their incidences are too rare to be statistically stable in studies with a small sample size. Therefore, as a first step in the MSCP project, we used the multicentre design to construct a uniform surgical outcome reporting platform in China, where the high volume of surgical patients enables rapid big data collection for patient safety studies. This practice of monitoring sporadic and rare events with prospective databases is in common with the UK Medical Research Council guidance for developing and evaluating complex interventions.32

A second difficulty in patient safety studies is the lack of concrete evidence in the process of intervention formulation. Unlike the well-established drug development process, which puts more effort into pretesting candidate chemicals than validating their effectiveness, a common practice for patient safety initiatives is to introduce methods that are supported by theoretical basis from other fields into the clinical setting,33 and often swiftly move into a validation phase that requires a great sample size for evaluating such rare ‘hard endpoints’.6 33 Once proven non-effective, clinical staff may get frustrated for further input. In the MSCP, the baseline data survey of safety status will form an evidence base for formulating the intervention schemes. By pointing out the most important safety issues and high risk patients, introduction of an evidence-based approach from validation of interventions to formulation of interventions enhances the value of data in the safety reporting systems.34 35 Besides, not every intervention item is necessary for all patients. It has been widely acknowledged that interventions that are based on local data can be more cost effective, and precisely tailored interventions can result in higher compliance among medical staff because of executive briefness and sense of involvement.16 36

Using information technology to provide a solid infrastructure for safety enhancing approaches is a distinguishable feature of the system-based MSCP. The combination of the safety intervention with safety outcome data not only enables an iterative process of developing and evaluating safety approaches37 but also provides a more intuitive picture to the care providers of the association between ‘what has been done’ and ‘what benefit to the patients is achieved’, so that they are more motivated for further devotions.28 The information transfer function will help coordinate several professional groups and strengthen the unity of the surgical team for a single patient. Particularly, by sorting out the safety roles of the frontline medical staff and the middle managers and hospital administrators25 in the ‘Graduated Confirmed Awareness’ subsystem, the ‘soft science’ of culture management is also established into the long-lasting hospital system, and the day-to-day flow of data within this system, from an management perspective, helps to replace the strenuous and occasional efforts for change and reduce the human labour for large-scale supervision.38

Demonstrating the effectiveness of an intervention (the goal of Stage 2) is never an end point in patient safety practice, as a number of studies have shown that any safety intervention, even for the simple surgical safety checklist or other robustly evidence-based practice, may fail in a complicated clinical context if it is not ideally implemented.36 39 This was the rationale for Stage 3, which in some sense is more practice oriented (rather than evidence-oriented in Stage Two) and mainly aims to formulate strategies to enhance the implementation quality and efficiency for wider external promotion. In fact, medical staff members are often overburdened with multiple work priorities. It has been indicated, both among clinicians and nurses,40 41 that workload can be adversely associated with patient safety. Therefore, measures are taken such as facilitating the data reporting and intervention implementation with electronic system and reducing the items and required time to a minimum that also guarantees patient safety. Nevertheless, balancing the strength of safety intervention and clinical acceptance and consistently reducing the burdens associated with implementation will be ongoing endeavours in the MSCP and any further work in patient safety.

There are reflections for the current work that should be taken as future orientations. Three elements of the clinical-microsystem are considered important for patient safety according to a three-dimensional model: the system, culture and technology.42 43 For the first element, we intend to integrate well-established approaches for preventing surgical complications into solid hospital systems to make the delivery more target sensitive and organised; however, the development process of the interventions does not include ergonomic analysis or any process analysis techniques like that; incorporation of which would help to rationalise the workflow and enhance the effectiveness.43 For the second element, though various measures are proposed to engage frontline clinicians and nurses, workplace culture is very complex in the clinical context, for which qualitative investigation methods such as personal interviews would have identified more in-depth points for improvement.44 For the third element, we leave a number of technique-related complications unintervened, especially in the specialty-specific complication category as they are out of the general scope of large-scale quality improvement, and developing techniques for them would be more challenging and require a higher level of evaluation process.38