Development of screening tools to predict the risk of recurrence and related complications following anal fistula surgery: protocol for a prospective cohort study.

INTRODUCTION: Postoperative recurrence and related complications are common and related to poor outcomes in patients with anal fistula (AF). Due to being associated with short-term and long-term cure rates, perioperative complications have received widespread attention following AF surgery. This study aims to identify a set of predictive factors to develop risk prediction models for recurrence and related complications following AF surgery. We plan to develop and validate risk prediction models, using information collected through a WeChat patient-reported questionnaire system combined with clinical, laboratory and imaging findings from the perioperative period until 3-6 months following AF surgery. METHODS AND ANALYSIS: This is a prospective hospital-based cohort study using a linked database of collected health data as well as the follow-up outcomes for all adult patients who suffered from AF at a tertiary referral hospital in Shanghai, China. We will perform logistic regression models to predict anal fistula recurrence (AFR) as well as related complications (eg, wound haemorrhage, faecal impaction, urinary retention, delayed wound healing and unplanned hospitalisation) during and after AF surgery, and machine learning approaches will also be applied to develop risk prediction models. This prospective study aims to develop the first risk prediction models for AFR and related complications using multidimensional variables. These tools can be used to warn, motivate and empower patients to avoid some modifiable risk factors to prevent postoperative complications early. This study will also provide alternative tools for the early screening of high-risk patients with AFR and related complications, helping surgeons better understand the aetiology and outcomes of AF in an earlier stage. ETHICS AND DISSEMINATION: The study was approved by the Institutional Review Board of Shuguang Hospital affiliated with Shanghai University of Traditional Chinese Medicine (approval number: 2019-699-54-01). The results of this study will be submitted to international scientific peer-reviewed journals or conferences in surgery, anorectal surgery or anorectal diseases. TRIAL REGISTRATION NUMBER: ChiCTR1900025069; Pre-results. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first large prospective cohort study of patients with anal fistula at a tertiary referral hospital in China.

A higher events per candidate predictive variable (≥20) will be applied, which can generally eliminate bias in regression coefficients for prediction models and guarantee a sufficient sample size for model development.

Candidate predictors will be identified from published and updated systematic reviews, expert opinions from Delphi surveys and univariable or multivariable logistic regression analysis.

The bootstrapping procedure will be applied for the internal and external validation of the prediction models.

A higher probability of missing data due to non-response bias may occur as many of the variables are collected through a WeChat questionnaire system.

Introduction

Anal fistula (AF) is a common perianal condition defined by a pathological epithelial tract that connects the anal canal or rectum and the surface of the perianal region, which is also regarded as a chronic stage of perianal abscess.1 Postoperative recurrence, defined as persistence or recurrence of AF symptoms, or the development of recurrent perianal sepsis or chronic AF within 6 months of surgery,2 3 is not only one of the consequences that can be related to a poorly performed surgical procedure but may also be due to the insidiousness of the disease. Our recently published meta-analysis based on 20 studies reported a recurrence rate of approximately 19% (95% CI 0.15 to 0.23) in patients undergoing AF surgery.4 Because of the high degree of difficulty of surgery for patients with high complex AF, the postoperative recurrence rate of these patients can be as high as 50%, and the failure rate of reoperation remains at 10%.5–8 It is considered one of the most difficult and complicated anorectal diseases.

A large number of studies have shown that the recurrence of AF is related to multiple factors, such as unclear diagnosis or failure to focus on the correct internal orifice, blind stump of fistula, incorrect method of seton, omission of branch of the fistula and poor drainage.9 10 Studies also reported that anal fistula recurrence (AFR) was associated with individual patient characteristics, such as history of enteritis, previous anal surgery, obesity and smoking.3 4 11 12 Li et al retrospectively analysed 1783 patients with AF receiving surgical treatment and found that the location of AF, previous perianal surgery, seton history and enteritis were independent risk factors for AFR.5 Recently, according to the evidence grading criteria based on Egger's P value, total sample size and between-study heterogeneity, we published a meta-analysis involving 20 studies with 6168 patients and concluded that high transsphincteric fistula, unidentified internal opening and horseshoe extensions were independent risk factors for AFR with high-quality evidence, while prior anal surgery, seton placement surgery and multiple fistula tracts were demonstrated to be risk factors for AFR with moderate-quality evidence.4

Factors influencing other perioperative complications related AF surgery including wound haemorrhage, faecal impaction, urinary retention, delayed wound healing and unplanned hospitalisation are also rarely reported. Therefore, there is an urgent need to develop risk prediction tools for the complete profile of risk factors for AFR and related complications.

Which patients with AF will be cured after surgery and which ones will not, are rarely investigated. The development of a prediction model for AFR following surgery to identify those patients with a higher risk of developing complications during follow-up would be of significant importance. First, surgeons can provide patients preoperatively with an estimated surgical cure rate according to the prediction models. Moreover, the current knowledge in the literature reporting potential predictive factors could help patients become familiar their individual risk factors and avoid modifiable ones to improve the cure rate, which has been well described and applied in the prevention of other diseases.13–16

However, to date, there are no effective screening tools to evaluate and predict the risk of recurrence or other adverse outcomes of AF. Therefore, the aim of the current study was to develop and validate multivariable prediction models that predict postoperative AFR and related complications.

Aims and objectives

The aim of this study was to develop risk prediction models for postoperative recurrence as well as other surgery-related complications in a prospective hospital-based AF cohort. A risk prediction model for perioperative complications will also be developed. A flowchart of prediction model development and assessment is provided in figure 1.

Figure 1

Flowchart of prediction model development and assessment.

The detailed tasks of this study are as follows:

Calculate the 3–6 months incidence of recurrence, and related complications in patients following AF surgery.

Establish the risk factors that significantly predict postoperative AFR and related complications based on the AF cohort in a tertiary referral centre.

Develop and validate the risk prediction models for postoperative AFR and related complications.

Considering the different scenarios for different surgical interventions, conduct stratified analyses based on surgery type. If possible, a risk prediction model will also be developed in relevant subpopulations, such as those only receiving fistulectomy or fistulotomy, which can account for more than 60% of our AF cohort populations.

We also examine the following two hypotheses:

Patient-related demographic characteristics, fistula and surgery-related factors are predictive of postoperative AFR and related complications as dependent variables.

The risk prediction models for postoperative AFR and related complications developed in our study have more than 70% discriminating power.

Patients and methods

Study design and participants

This study is a single-centre, prospective observational study on a hospital-based cohort enrolled at a tertiary referral centre in Shanghai, China.

Eligibility criteria

The enrolment of the cohort subjects was initiated in June, 2019. All subjects who will undergo surgical intervention for AF will be included. All operations will be performed by a group of colon and rectal surgeons at Shuguang Hospital, a regional tertiary referral centre. The exclusion criteria were age <18 years, non-cryptoglandular fistula (eg, AF due to inflammatory bowel disease, HIV, malignant cancer or obstetrical trauma) and rectovaginal or rectourethral fistula. The electronic medical records of the included subjects were complete.

Trained clinical investigators are collecting data in several categories, including baseline demographics, laboratory examinations, clinical data, imaging findings and follow-up information 3–6 months postoperatively. Planned clinical reviews or electronic surveys are conducted during hospitalisation and every 0.5–3 months after discharge for 6 months.

Data collection

The research team comprised a principal investigator and 5–8 anorectal surgeons who were trained and supervised by the Ethics Committee of Shuguang Hospital. Written informed consent was obtained from all patients. The investigators did not intervene in any aspects of patient surveys at any stage of data collection and follow-up. Data were collected using a convenient follow-up system supported by Empower Electronic Data Collection (EDC) (Solutions, Boston, Massachusetts, USA). This electronic system introduces a machine learning algorithm, through which we can use the data already entered in the Empower system to train the algorithm model and let the system itself develop quality control algorithms, validate the entered data and identify missing or suspicious data. Finally, the data manager will check the missing or suspicious data, confirm their completeness and ask the data manager to provide additional data when necessary. Furthermore, an automatic reminder follow-up function also plays a pivotal role during the whole follow-up period.

Patient and public involvement

Patients and the public will not be involved in the development, design, conduct or reporting of the study. The general results will be disseminated to participants through public education during follow-up.

Clinical outcomes

The primary study endpoint is postoperative recurrence following AF surgery defined as the persistence or recurrence of AF symptoms, or the development of recurrent perianal sepsis or chronic AF within 3–6 months of surgery.2 3 17 The second end point is a composite outcome of postoperative comorbidities or any equivalent events including AFR, wound haemorrhage, faecal impaction, urinary retention, delayed wound healing or unplanned hospitalisation associated with AF surgery. Outcomes were ascertained by the treating clinician combined with outpatient medical records or patient self-reports.

Selection of predictor variables

Candidate variables for the prediction model of the composite outcome of postoperative comorbidities will be screened according to the following pre-set criteria: (1) prior clinical knowledge; (2) results from a systematic review updated in November 2019 based on our published one4 with sufficient evidence to include them as predictive variables in the risk model for AFR as is demonstrated below; or (3) agreed upon by a group of anorectal surgeons or experts for their clinical relevance using a two-round Delphi survey. We initially identified the following covariates as relevant candidate variables based on systematic reviews as well as clinical knowledge and/or relevance. The determination of all other candidate variables is based on the results of post-hoc analysis using univariable or multivariable survival analyses with a threshold of p<0.05.

Factors identified from the systematic reviews and Delphi survey (manuscript under review), will be measured at baseline. These include factors involving the identified significant risk factors that are reported in our meta-analysis and are presented as follows:

Prior anal surgery.

Seton placement surgery.

High transsphincteric fistula.

Internal opening unidentified.

Horseshoe extensions.

Multiple fistula tracts.

Some of the demographic factors and surgery details will also be collected due to limited power in the literature reviews and some non-significant potential factors (eg, smoking or alcohol use) may be risk factors and are also included as follows:

Gender.

Age.

Smoking.

Alcohol use.

Diabetes mellitus.

Obesity.

Preoperative seton drainage.

High internal opening.

Postoperative drainage.

Supralevator extensions.

Data on other factors, such as laboratory examinations and MRI parameters (height of the internal openings, height and number of fistula, etc.), will also be collected. Moreover, other factors, such as chronic steroid therapy, diverting stoma, the surgeon's level of training, postoperative bowel confinement and antibiotic prophylaxis reported in previous literature, were selected for regression analysis as well. In addition, relevant factors from the expert-opinion survey were also assessed including the number of prior AF surgeries, the types of surgery performed (such as staged fistulotomies, endorectal advancement flap and ligation of the intersphinteric fistula tract with and without seton drains), some nutrition parameters and immunomodulation medication use.

Categorisation of potential predictors

We can code categorical predictors as ‘factor’ variables, coding them as dummy variables. For example, smoking is coded originally as ‘1’ for never smoker, ‘2’ for past smoker and ‘3’ for current smoker and never smoker was selected as the reference category. A similar approach can be used with alcohol use.18 Continuous variables formally should be measured with an interval or ratio scale, and should be able to take any value in a range. We treat ordered variables as linear which is generally reasonable for prediction. In other cases, continuous predictors can be grouped with meaningfulcategorisation; for example, body mass index (BMI) can be classified based on internationally recognised categories (ie, underweight, normal weight, overweight and obesity).19 Based on previous experiences, we will derive some predictors based on the responses of the surveys. However, in case some subjectivity in the classifications of these predictors may occur, sensitivity analyses will be performed to examine the robustness of our definitions during model development and validation.

Study quality control for the prediction models

Based on the summary of methodological quality and the developmental stage of prediction models by van Oort et al, we are developing predesigned criteria for the quality control of our prediction models, which can allow us to conduct the study and report the results more rigorously.20 21 The methodological checklist of the study is presented in the online supplementary material.

Missing data

Candidate predictors with more than 60% missingness will be excluded. For those with less than 60% missingness, multiple imputations are to be performed by imputing 20 complete data sets using multivariate normal regression,22–25 which can reasonably approximate the true distributional relationship between the missing values and the available ones.26 Among various multiple imputation approaches, fully conditional specification (FCS) and multivariate normal imputation (MVNI) are preferred, because they have been proven to be generally less biased than complete-case analysis. They can both generate similar results in the presence of either binary or ordinal variables that are not generally normally distributed.26

Statistical analysis for model derivation

Logistic regression will be applied to develop our prediction models for the binary outcomes. All data processing and statistical analysis will be performed using EmpowerStats software (www. empowerstats.com; X&Y Solutions, Inc., Boston, Massachusetts, USA) and the statistical software package R (R Foundation, Vienna, Austria).

We will first study the association between each potential variable and the outcome based on univariable analysis. Variables are considered further for multivariable regression modelling when they are associated with a p- value <0.20. Normality or linearity will be evaluated for the continuous predictors. Fractional polynomials are advocated for associations between the continuous predictors and the outcome for non-linear relationships.27 28 We will perform backward stepwise selection with p<0.001 as the inclusion threshold and p>0.05 as the exclusion threshold for each imputed data set.

Predictors that appear in the imputation models with an inclusion fraction of ≥50% are qualified for the final multivariable model. Although there is no consensus regarding the optimal method for selecting predictors for inclusion, backwards elimination is generally considered as the preferred procedure as reported by Mantel et al.29 A forward stepwise procedure will also be performed to repeat the analysis to test the robustness of the models. Overall, regression coefficient estimates of the models will be generated with the combination of the imputed datasets based on Rubin’s Rules, while taking into account uncertainty in the imputed values.22 23 25 Collinearity will also be assessed, which refers to the fact that predictors can have strong correlation with each other, defined as correlation coefficient >0.8, or variance inflation factor >10.30 Then we will examine the interactions among the regression models.

Prediction model performance assessment

Prediction models will be developed with a random sample of 60% of the AF cohort as the derivation cohort, and then validated with the remaining sample of 40% of the cohort as the validation cohort. The predictive performance in the derivation and validation cohort will be evaluated and reported by examining measures of predictive accuracy, discrimination and calibration. Nagelkerke’s R2 and the Brier score will be used for the measurement of predictive accuracy.31 32 The discriminative ability of the prediction models is evaluated using several statistics, which are according to the discriminative and calibration ability in both the derivation and validation AF cohorts. Model discrimination is the ability of the models to differentiate between high-risk patients and low-risk patients (having high or low risk of AFR or surgery-related complications). This will be assessed via Harrell’s concordance statistic (C-index).33 The calculation of the C-index will be performed in each of the 20 imputed data sets, and then averaged based on Rubin’s rule.34 The model is interpreted as having no discriminatory ability when a value of C-index is 0.5, and has perfect discrimination when a value of the C-index is 1.0.33 Calibration implies the agreement between the predicted outcomes and the observed outcomes, which is evaluated with the Hosmer-Lemeshow goodness-of-fit test in all imputed datasets presented with calibration plots.35 Calibration-in-the-large, which is defined as the agreement between mean observed outcomes and mean predictions, will also be assessed for calibration.36

Internal and external validation of the prediction model

To make the prediction models reproducible, we must perform internal validation. The bootstrapping technique, as one of the most attractive resampling techniques, is a mostly applied validation method that seems to be most efficient for obtaining stable optimism-corrected estimates.33 37 It has been reported that bootstrap validation is a feasible technique for most prediction models with at least a 500 bootstrap resampling procedures using Harrell’s validation function, which can adjust the developed models for overfitting.38 We will also apply temporal validation as external validation using a more recent patient with AF cohort.39

Sample size

Since there are no widely accepted methods for the estimation of the sample size requirements to develop the risk prediction models, the size of this AF cohort will be calculated to have 20 events per candidate predictive variable (EPV, defined as the ratio of the number of individuals with the outcome event to the number of candidate predictors), which can generally eliminate bias in regression coefficients for prediction models with low-prevalence binary predictor development (the estimated recurrence rate <20%) and adequately power the logistic regression models.40 41 According to the findings by Ogundimu et al,42 a higher EPV (≥20) can generally eliminate bias in regression coefficients for prediction models with low-prevalence binary predictor development. Then we estimate 400 events allowing for 20 predictor variables (EPV=20). Considering a 5%–20% recurrence rate, we assume that at least 4000–8000 patients should be recruited for model development. In addition, surgery type and fistula type-stratified analyses will also be performed to examine the different effects of these factors on disease recurrence or other related complications in each subgroup. Risk prediction models, if possible, can also be developed in those subpopulations. A cohort size with more than 4000–8000 patients will provide sufficient power to perform those analyses and develop prediction models in those subgroups.

Follow-up and methodological quality control

The application of WeChat questionnaires to collect data will inevitably increase the probability of missing data. However, we have made some predesigned countermeasures. For example, we have set up follow-up reminders via the WeChat questionnaire system. Moreover, every week two trained clinical fellows cross-check the data, and will contact the respondents by phone or WeChat about the missing contents, which can minimise the missing data and loss to follow-up rate.

A limited number of studies have identified specific criteria for quality control in a prediction model, but we have strictly adhered to the guidelines for the reporting of developing studies, validating multivariable clinical prediction models as is reported in the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement to ensure methodological rigour.43 All issues have been addressed in this study in the online supplementary material.

Ethics and dissemination

The results of this cohort will be submitted to international scientific peer-reviewed journals or conferences in surgery, anorectal surgery or anorectal diseases.

Discussion

In this study, we plan to develop internally validated models for the prediction of the recurrence as well as postoperative complications among patients with AF. The models will be developed based on a large AF cohort in a hospital-representative linked database with validated clinical information. The collected variables include WeChat questionnaires, clinical, laboratory and imaging findings, and follow-up information, all of which are routinely being gathered at the time of enrolment.

According to the existing knowledge and systematic reviews, it is highly plausible that a number of patient, fistula and surgery-related characteristics (eg, patient-related variables such as gender, age, diabetes mellitus or obesity);3 44 45 lifestyle factors such as smoking and alcohol abuse;46 47 fistula-related factors such as the number of fistula tracts, horseshoe extensions, classification and location of fistula; and surgery-related variables such as prior anal surgery and postoperative drainage46 48 49 are easily ascertainable before surgery may predict AFR. Similar risk prediction models exist in other diseases, such as the Framingham risk score model to predict cardiovascular disease risk50 and the Korean Crohn's disease prediction (KCDP) model to predict the clinical course of Crohn's disease.51 Until now, risk factor investigations of predictors of perioperative surgery-related complications have been limited to assessments of single predictors with small sample sizes. Risk prediction models can help inform surgeons regarding high-risk patients with AF based on the overall risk factors. The primary purpose of this study was to develop two risk prediction models to assist surgeons in identifying patients with AF scheduled for surgical treatment who are at higher risk of developing recurrence and surgery-related complications. The predictive models will help both clinicians and patients identify the risk of complications after AF surgery in advance, and perform the interventions necessary to reduce the risk of surgery-related complications and the personal and social financial burden brought about by those complications. Accurate risk prediction models are especially instructive for the development of the optimal surgical plan to achieve optimal surgical outcome.

Our study has several strengths. To the best of our knowledge, this is the first study with the primary aim of developing and internally and externally validating multivariable prediction models for AFR and related complications following AF surgery. Multidimensional clinically useful candidate predictors will be fully examined from a variety of sources including our published and updated systematic reviews, expert opinions from Delphi surveys and univariable or multivariable logistic regression analysis. Second, we will apply the internal and external validation of the prediction models using the bootstrapping procedure. Third, multiple imputation will also be used to treat the missing data. Finally, our study is a prospective cohort study with an adequate follow-up period which can minimize certain forms of bias.

Our study also has limitations. As many of the variables are collected through a WeChat questionnaire system during hospitalisation and follow-up, a higher probability of missing data due to non-response bias may occur. To address this issue, we regularly send reminders to those who do not respond after discharge. Second, although we will investigate a series of potential predictors, some more potential predictors will not be involved or not collected in the current study, such as data related to postoperative nursing strategy and outpatient follow-up frequency. Moreover, bias may also result from the single-centre recruitment of our study and will be improved through multicentre recruitment in the future. Finally, one key potential issue that needs to be considered a priori is the variable recurrence rate depending on the risk factors included and identified in the testing and validating cohorts since this may affect the C-index. It is possible that a lack of key variable inclusions in the models may result in decreased discriminatory ability. This is a function of the database and points of interest included that may need to be maximised before proceeding with development of the testing model.

The newly developed risk algorithms may have significant applications in clinical practice by helping recommend an optimal surgical approach for a specific patient with AF, as well as intensive perioperative care, education, and the timely assessment and discussion of the need for interventions among those most at the highest risk of developing recurrence or surgery-related complications. The models will specifically identify patients with AF who are likely to develop recurrence or related complications following AF surgery to offer a quantitative evaluation of the risk. Moreover, the models will also provide reference information for preventing recurrence, reducing the rate of recurrence after operation and intervening in high-risk factors in the early stage.

In summary, this study protocol summarises the design of development and validation studies for a risk screening tool in patients receiving AF surgery. The results from this study will be interpreted for the purpose of clinical decision making. The models to be developed in this study could be used to make new recommendations for perioperative patients with AF.