LIAISON® Calprotectin for the prediction of relapse in quiescent ulcerative colitis: The EuReCa study.

INTRODUCTION: Fecal calprotectin (FC) is established as a diagnostic marker to differentiate between inflammatory bowel diseases and non-inflammatory conditions. Furthermore, it may be effective in monitoring response to treatment, and to predict relapse during maintenance therapy.
DESIGN: This was a prospective longitudinal study carried out in Italy, France and Spain. The primary objective was to correlate the LIAISON® Calprotectin assay measurements to quiescent ulcerative colitis (UC) or relapse as assessed by clinical data. Patients were assessed every 3 months for 12 months, and at 18 months.
RESULTS: The last FC measured prior to relapse was the variable that predicted relapse in a statistically significant manner. With a 62.3 μg/g cut-off the area under the curve was 0.619, and the sensitivity was 62.9% (95% Confidence Interval [CI] 44.9%-78.5%) and specificity 63.0% (95% CI 53.1%-72.1%). Using machine learning methods, the last FC measurement was shown to have the largest impact in predicting relapse. An algorithm was developed that included other variables available following a clinician's visit, which resulted in an area under the curve of 0.754 for predicting relapse.
CONCLUSION: In the present study FC measured by the LIAISON® Calprotectin assay on the visit before relapse is predictive of relapse in patients with quiescent UC. In a proof of concept, the accuracy of prediction can further be improved including other variables in an algorithm developed by machine learning. TRIAL REGISTRATION: The trial is registered at clinicaltrials.gov with reference number NCT05168917.

Summarise the established knowledge on this subject

Fecal calprotectin (FC) is used in the diagnosis of inflammatory bowel diseases to differentiate from non‐inflammatory conditions.

What are the significant and/or new findings of this study?

The effectiveness of FC to predict relapse in quiescent patients during maintenance therapy was explored.

The last FC measured prior to relapse predicted relapse in a statistically significant manner.

A machine learning algorithm was developed with data from routine clinician's visit, which resulted in improved relapse prediction.

INTRODUCTION

Fecal calprotectin (FC) is a calcium binding protein belonging to S100 protein family. It represents up to 60% of the soluble protein content within the cytoplasm matrix of neutrophil granulocytes. In patients with inflammatory bowel diseases (IBD), the modification of intestinal mucosa, and the increase in permeability allows neutrophils to reach the intestinal lumen increasing the FC concentration in stool samples.

FC is well established as diagnostic marker to differentiate between IBD and non‐inflammatory conditions, such as irritable bowel syndrome (IBS). , Furthermore, it may be effective to monitor response to treatment, and to predict relapse in IBD patients during maintenance therapy. , , , , , , FC also well correlates with histological activity in ulcerative colitis (UC).

The exact timing for monitoring UC activity after the start of a new treatment is not established. Additionally, indications for monitoring quiescent disease in order to prevent or detect disease flares early are not well understood. The last European Crohn's and Colitis Organization (ECCO) guidelines suggest to use FC, together with clinical parameters and other biomarkers of inflammation, such as C‐reactive protein (CRP) every 3–6 months, however very low evidence supports this recommendation.

Studies have been conducted on the role of FC to predict further relapse overtime in patients with quiescent UC. Kostas et al. found that FC value higher than 261 μg/g, had a strong predictive value for the discrimination of future relapses versus maintenance of remission, although this was a retrospective study on a small number of patients. Evidence rising from prospective studies on the prediction of future relapses by FC is still scarce. Here the results of the European relapse calprotectin study (EuReCa) are presented.

METHODS

Study subjects

This was a prospective longitudinal study. The primary objective was to correlate FC values, measured by LIAISON® Calprotectin (Diasorin), to quiescent UC and to disease activity (relapse) as assessed by clinical data and, when available, by endoscopy. Secondary objectives were to assess if a 3 months interval between two measurements of FC in quiescent patients is an adequate length of time to detect ensuing relapse. Based on the assumption of a relapsing occurrence rate of 15%–25% with significant level at 95% and power at 80%, the estimated sample size was 200, which was met by total enrollment numbers, but short for the eligible patients (Figure 1).

FIGURE 1

Flow diagram of enrolled patients

Adult patients (age ≥18) in clinical and endoscopic remission, confirmed by proctosigmoidoscopy at enrollment, were considered eligible for this study. Patients had previously been diagnosed with left‐sided colitis or pancolitis, and were receiving maintenance therapy as per current medical practice. Clinical and endoscopic remission of the patients was defined as a complete Mayo score <2. Exclusion criteria were microscopic colitis, Crohn's disease, limited proctitis (<5 cm from the anal verge), any severe chronic disease affecting the possibility to comply with the study protocol (i.e. severe cardiovascular disease, renal or liver failure, neurologic disease, hematological disease, and mental disorder), anticoagulant therapy, and pregnancy or lactation.

After inclusion, all enrolled patients were followed up every 3 months for 12 months (3, 6, 9, and 12 months alternating visits by telephone contact), and subsequently at 18 months. Three different IBD referral centers from Italy, France and Spain participated in this study. At 6 and 12 months, patients were assessed for disease activity by calculating a partial Mayo Score (number of stools exceeding the normal number, rectal bleeding, physician's global assessment), and were requested to collect one fecal sample for FC measurement. At 3, 9 and 18 months, patients were followed up by a remote visit in which a partial Mayo score was calculated. At these time points, patients were asked to send a fecal sample to the investigating center to be tested for FC.

Loss of clinical remission was defined as an increase in the partial and complete Mayo score (≥2) after exclusion of infectious entero‐colitis as recommended by the ECCO Guidelines. Endoscopy was performed in most patients to confirm relapse. Patients were asked to provide a fecal sample at relapse for FC testing.

At each study visit, any adverse event occurring was reported according to the current Good Clinical Practice guidelines. Serious adverse events were reported to the sponsor by the investigator within 24 h of awareness. The clinical performance study was conducted in accordance with the ethical guidelines of the 1975 Declaration of Helsinki and local regulations. Each subject signed an informed consent form. The study was approved by the local ethical committee in each participating center: 1) Comitato Etico Indipendente del IRCCS Istituto Clinico Humanitas—approval date 21 February 2017, 2) Comité de Protection des Personnes Ile de France IV—approval date 26 June 2017 and ANSM Direction des dispositifs médicaux de diagnostic et des plateaux techniques—authorization date 18 July 2017, 3) Spain Comite de Etica de la Investigación con medicamentos (CEIm) Grupo Hospitalario Quiron en Barcelona—approval date 28 February 2017. Data were collected in an electronic certified case report form, and a regular monitoring for quality of data and procedures by an independent certified company was performed. The trial is registered at clinicaltrials.gov with reference number NCT05168917.

Fecal calprotectin testing

Calprotectin was measured using the automated LIAISON® Calprotectin assay (DiaSorin), a quantitative, chemiluminescent, sandwich immunoassay with a measuring range of 5–8000 μg/g, which is comparable to automated tests by other manufacturers (BÜHLMANN fCAL® turbo 20–8000 μg/g, Inova QUANTA Flash® Calprotectin 16.1–3500 μg/g). All stool samples (minimum quantity 50 g) collected were frozen (−20℃) and sent to a centralized lab (DiaSorin Inc.) for testing on a regular basis (every month or when an acceptable number of samples was available). Extractions were performed using the weigh method according to the manufacturer's instructions for use. The samples were coded, and the technician performing the testing was blinded to the patient information. Once all the testing was performed results were returned to the investigator of the relevant clinical site, where the sample codes were broken and the results analyzed. If local testing occurred on samples collected for the study, only data from centralized testing were used for the clinical performance study analysis.

Statistical analysis

MedCalc v20 was utilized for most analyses presented. The nonparametric Mann‐Whitney rank test was used to assess significant differences between groups, and χ2 was used to assess frequencies. R was used to explore significant predictive parameters by Wilcoxon analysis to assess if baseline and last measurements and their ratio differ between relapsed and quiescent patients. Logistic regressions were used to determine odds ratios. The adequate threshold for FC was assessed by ROC. Once the threshold was set, a Kaplan‐Meier survival curve was used to determine the ability of the last measured FC value to predict relapse. With regard to diagnostic performance, clinical sensitivity, specificity, negative and positive predictive values were computed comparing to the clinical diagnosis.

Machine learning

Non‐prespecified post‐hoc analysis was explored by machine learning tools to optimize a prediction model of relapse using additional features (or variables) collected at the clinical visits, and to assess whether a machine‐learning based algorithm that includes FC and other variables could be accurate in predicting UC relapse better than FC alone. Features examined for the machine learning mode included patient general information, Mayo partial scores, blood test panels including CRP, drug history, previous disease location, and basal and last FC. Mayo endoscopic scores were excluded. Logistic regression was used for training an algorithm in a supervised data classification. The purpose of the algorithm was to build a linear probabilistic model of classification. In the training phase, the logistic regression algorithm used 80% of the dataset to elaborate the algorithm. The algorithm was then tested on the remaining 20% of the set, augmented by Synthetic Minority Oversampling TEchnique (SMOTE) or Adaptive Synthetic (ADASYN) methodologies , in order to test the algorithm stability and convergence with data increase.

RESULTS

The rate of relapse was 24.5%, consistent with expected numbers. In Table 1 the baseline characteristics of the quiescent and relapsed patients are presented. No significant differences were observed for any of the parameters between the two clinical outcome groups. The only slight statistically significant difference was the last FC measurement (last before relapse or last of the study) at p = 0.0348.

TABLE 1

Characteristics of the enrolled subjects according to their relapse status

	Quiescent N = 106	Relapsed N = 35	p value
Sex			0.54
Male	64 (59%)	23 (66%)
Female	41 (38%)	12 (34%)
Age (years)			0.14
≤35	25 (23%)	10 (29%)
36–45	29 (27%)	12 (34%)
46–55	22 (20%)	9 (26%)
56–65	19 (18%)	3 (9%)
>65	13 (12%)	0 (0%)
UC diagnosis			0.32
Left‐sided colitis	76 (70%)	29 (83%)
Pancolitis	30 (28%)	6 (17%)
Gut location
Ileum	2 (2%)	0 (0%)	0.15
Caecum	22 (20%)	3 (9%)	0.11
Ascending	33 (31%)	11 (31%)	0.92
Transverse	38 (35%)	10 (29%)	0.47
Descending	47 (44%)	19 (54%)	0.27
Sigmoid	104 (96%)	34 (97%)	0.81
Rectum	101 (94%)	34 (97%)	0.42
Severity of past disease			0.49
Mild	15 (14%)	3 (9%)
Moderate	33 (31%)	16 (46%)
Severe	14 (13%)	3 (9%)
Inactive	4 (4%)	2 (6%)
Not available	42 (39%)	11 (31%)
Medications
Mesalazine	74 (69%)	22 (63%)	0.49
Corticosteroids	7 (6%)	5 (14%)	0.18
Azathioprene	16 (15%)	6 (17%)	0.84
Anti‐TNF	20 (19%)	6 (17%)	0.73
Other	25 (23%)	9 (26%)	0.87
Other
Initial diagnosis (years prior)	7.6 (4.0–13.2)	6.1 (4.0–10.8)	0.45
CRP (mg/dl)	0.13 (0.08–0.27)	0.16 (0.08–0.27)	0.64
FC
Baseline (μg/g)	34.5 (10.7–89.8)	35.6 (10.8–129)	0.56
Last or last before relapse (μg/g)	38.7 (10.7–132)	100 (15.4–457)	0.0348
Last FC from relapse (days)	‐	78 (57–104)
FC during relapse		784 (538–1730)

Note: Numbers represent frequencies (%) or median with interquartile ranges. The bold p values are for chi square of frequencies of multiple parameters.

Abbreviations: CRP, C‐reactive protein; FC, fecal calprotectin; UC, ulcerative colitis.

The variables were analyzed considering both the occurrence or not of relapse, and the time to either relapse or exit from the trial. Variables of most interests at baseline, and at the last measurement prior to relapse interest were FC and CRP: a nonparametric Wilcoxon test was performed to determine whether the baseline and the last FC or CRP prior to relapse, and the ratio of baseline to last, differed between patients experiencing relapse and those not relapsing (Table 2). Of the six variables, only the last FC measured prior to relapse showed a statistically significant difference. Next, a more powerful test of association used logistic regression to test the three FC variables (Table 2). CRP variables were not further studied as they did not approach significance in the initial test. According to the logistic regression results the baseline FC remains non‐significant, the ratio of the last FC to the baseline becomes marginally significant, and the last FC remains statistically significant in predicting relapse (Table 3).

TABLE 2

Nonparametric Wilcoxon analysis results to determine which variables differed between patients experiencing relapse, and those not relapsing, and logistic regressions to test the association of FC variables to the relapse outcome

Variable	Test statistic (W)	p value
Baseline FC	1701	0.9394
Last FC	1253	0.0188
Baseline/last FC ratio	1369	0.0782
Baseline CRP	1594	0.6495
Last CRP	1806	0.6535
Baseline/last CRP ratio	1867	0.4446

Abbreviations: CI, Confidence Interval; CRP, C‐reactive protein; FC, fecal calprotectin; OR, Odds Ratio; SE, Standard Error.

TABLE 3

Impact of variables on the AUC generated by machine learning models

Model	AUC
All variables	0.589
All but Mayo partial	0.590
All but last FC	0.460
	AUC	Precision	Recall
Final model	0.754	1	0.25

Note: The final model contains FC, age, length of disease, firstFC/lastFC, number of drugs, Mayo partial scores, disease location, past disease severity. Precision minimizes false positives, while recall minimizes false negatives. The final model was tested with augmented datasets by different methods. Low SD shows the stability and the consistency of selected model.

Abbreviations: ADASYN, Adaptive Synthetic; AUC, area under the curve; FC, fecal calprotectin; SD, Standard Deviation; SMOTE, Synthetic Minority Oversampling TEchnique.

Receiver operating characteristic analysis was performed for the FC at relapse, and the last FC prior to relapse (Figure 2a,b). The associated criterion to the Youden index for the last FC measured at the visit before relapse was >62.3 μg/g, which gave 62.9% (95% CI 44.9%–78.5%) sensitivity and 63.0% (95% CI 53.1%–72.1%) specificity, with an area under the curve (AUC) of 0.619 (95% CI 0.534–0.699, p < 0.05) (Figure 2b). With a disease prevalence of 24.5% the positive predictive value was calculated to be 35.5% (95% CI 27.9%–43.9%) and the negative predictive value 84.0% (95% CI 76.9%–89.2%). In contrast, the FC measurement at relapse exhibited an AUC of 0.942 with an associated criterion of >454 μg/g, the sensitivity and specificity were 79.4% and 94.4%, respectively. Kaplan‐Meier survival functions were fitted using the last FC with a cut‐point of 62.3 μg/g to model time to censoring. This gave a statistically significant separation. The survival functions are shown in Figure 3. The two FC groups have similar relapse rates until approximately 100 days, after which the curves diverge significantly (p = 0.0093). A Cox proportional hazard model using the last FC measurement was statistically significant showing 0.14% increased hazard per unit increase of last FC (p = 0.0029).

FIGURE 2

Receiver operating characteristic analysis with relapse as the outcome for FC measurements (a) at the relapse visit with associated criterion at >454 μg/g (79.4% sensitivity and 94.4% specificity), and (b) at the visit prior to relapse (last FC) with associated criterion at >62.3 μg/g (62.9% sensitivity and 63.0% specificity). FC, fecal calprotectin

FIGURE 3

Kaplan‐Meier survival functions were fitted using the last fecal calprotectin measurements with a cut‐point of 62.3 μg/g and relapse as an outcome. The curves diverge significantly after approximately 100 days (p = 0.0093)

Machine learning data analysis was explored to enhance the prediction capability of FC together with other features that can be collected at a physician's visit. Different approaches were explored (logistic regression, random forest, xgboost), , , but ultimately logistic regression was selected as supervised algorithm for data classification in a binary probabilistic model (relapse patient or no relapse patient) due to the amount of data from the clinical trial. The entire dataset was split randomly: 80% of the dataset was used in the training phase, and the remaining 20% was used to test the model. The results of the model were then subjected to an ex‐post analysis in order to identify the features with greater weight in the prediction of patients relapse with the SHAP library which allows to sort the variables according to their impact on the prediction and to consequently explain the black box of the algorithm. Figure 4 shows the impact of each feature on predicting relapse while Table 3 summarizes the AUC of various models: the importance of carefully choosing features to include in the model is emphasized by the substantial difference in AUC for the model with all features. In addition, the drop in AUC is markedly larger when the last FC measurement is removed, compared to removing partial Mayo scores which underlines the importance of the last FC measurement over partial Mayo scores in predicting relapse. Table 3 also shows the final model results containing only selected features. Precision is high due to an unbalanced dataset on dependent variable (the no‐relapse group as the majority class, the relapse group as the minority one).

FIGURE 4

Impact of the features on the predicted patient outcome in a machine learning model. The quantification of the contribution that each feature brings to the prediction made by the model is express as SHAP values (SHapley Additive exPlanations)

The dataset was then augmented by synthetic data injection with SMOTE or ADASYN methodologies in order to test the algorithm stability and convergence with data increase, and to handle the dataset imbalance between relapse and no‐relapse patients (25% relapse and 75% no relapse). , The algorithm was launched on 10 different datasets randomly augmented with the two methods mentioned above (44% relapse and 56% no relapse) to obtain 10 AUC (Table 3): as can be seen from the Standard Deviation, the model maintains intrinsic consistency and stability as the analyzed data increases.

DISCUSSION

The primary objective of this clinical trial was to assess the performance of FC values, measured by the LIAISON® Calprotectin in predicting relapse in a population of patients with quiescent UC. In terms of FC measurement, the performance of our model showed about 63% sensitivity and specificity, which is consistent with the fact that FC is not specific to UC, and FC values between 50 and 250 μg/g may usually predict relapse in about 50% of patients, as Dulai et al. showed recently, although in a different patient setting. For the latter values they propose a confirmatory colonoscopy, which can be avoided for values <50 μg/g or >250 μg/g.

The results of the present study resemble the data by Nakarai et al. (AUC = 0.65 with a sensitivity of 71%, and specificity of 60%) with similar cut‐offs (62.3 μg/g here vs. 75 μg/g). However, the study of Nakarai et al. consisted of a single FC sample at baseline, and subsequent relapse was assessed at bimonthly follow‐ups over 2 years, while in this study FC was measured every 3 months and the prediction window was thus shorter. Therefore, the comparison of the two studies is merely a point of reference. The data presented here are divergent from what was observed by Kostas et al., where baseline FC seems to predict relapse more reliably. However, the latter study had fewer UC patients (N = 36), was retrospective, and included quiescent patients that had high baseline FC values, which were absent in this study (only global Mayo of 0 and 1 were included here), as evidenced by the low mean baseline FC levels of the relapsed group (35.6 μg/g) here, compared to the much higher median baseline FC level of 481 μg/g. While direct comparison of FC values is difficult to make due to the lack of standardization across the different tests, the values and patterns were different. Other parameters such as CRP, and monocyte count have been previously suggested as predictive measurements, however we did not observe significant contribution from either. ,

The last FC measurement was found to be the best predictor of relapse in this study population where FC was measured every 3 months. FC measurements 3–6 months prior to relapse are stronger predictors compared to earlier measurements (data not shown) thus suggesting that a 3–6 months interval could be the most appropriate way to monitor our patients with UC in order to predict relapse before the symptoms occur. Both ECCO guidelines and the STRIDE Consensus suggest an interval of 3 months for FC monitoring, , , yet neither recommendation is supported by solid data. This study is the first to provide evidence for this monitoring schedule. Another important piece of information is that the ratio between baseline and last FC measurement can also significantly predict the risk of relapse, emphasizing the importance of looking at FC changes over time, in addition to the value at a definite timepoint, to efficiently monitor patients, as it is usually done in clinical practice.

FC showed to be the strongest predictor of relapse, however, as a standalone, the performance was suboptimal, which may not be unexpected due to the complexity of the disease. Including other variables or features in a logistic regression model using machine learning successfully improved the prediction capabilities; the variables added in this model can be obtained from the patient's records, and do not require further testing. This exploration of a machine learning model is a proof of concept, which needs to be confirmed and refined in larger prospective studies. However, this could be implemented to improve prediction by FC together with other individual patient characteristics to approach each patient in more personalized manner in the near future, specifically by transforming the model into a user‐friendly application to be used for patient monitoring.

This study had the following strengths: all the patients had FC measurement at the same timepoints (every 3 months); therapy was not changed based on the results of FC, giving a pure observation on the predictive role of FC without any bias related to the investigator's intervention. The limitations of this study were the sample size based on the relapse rate was not met by the number of eligible patients (power 80%, significance 90%), and the lack of endoscopic and histological data at the intermediate timepoints, that would have allowed examine the correlation of FC with histological/endoscopic changes over time. However, this study design reflects current clinical practice. The limitations on the post‐hoc machine learning model were the low number of subjects, and the unbalanced ratio of relapsed versus non‐relapse patients, which is however due to the expected prevalence. Being a proof of concept, it thus sets the stage for future work.

CONFLICTS OF INTEREST

Fabrizio Bonelli and Mariella Calleri are employees of DiaSorin the manufacturer of the LIAISON® Calprotectin test. Claudia Zierold is a consultant to DiaSorin. Employees and consultant of DiaSorin participated in the study design, data collection, data interpretation, and in the preparation of the manuscript for publication. Gionata Fiorino received consultancy fees from Ferring, MSD, AbbVie, Takeda, Janssen, Amgen, Sandoz, Samsung Bioepis, Celltrion. Silvio Danese has served as a speaker, consultant, and advisory board member for Schering‐Plough, Abbott (AbbVie) Laboratories, Merck and Co, UCB Pharma, Ferring, Cellerix, Millenium Takeda, Nycomed, Pharmacosmos, Actelion, Alfa Wasserman, Genentech, Grunenthal, Pfizer, Astra Zeneca, Novo Nordisk, Cosmo Pharmaceuticals, Vifor, and Johnson and Johnson. Laurent Peyrin‐Biroulet has served as a speaker, consultant, and advisory board member for Merck, Abbvie, Janssen, Genentech, Mitsubishi, Ferring, Norgine, Tillots, Vifor, Hospira/Pfizer, Celltrion, Takeda, Biogaran, Boerhinger‐Ingelheim, Lilly, HAC‐Pharma, Index Pharmaceuticals, Amgen, Sandoz, Forward Pharma GmbH, Celgene, Biogen, Lycera, Samsung Bioepis, and Theravance. Miquel Sans has served as a speaker, consultant, and advisory board member for UCB Pharma, Ferring, Falk, Millenium Takeda, Pfizer, Astra Zeneca, Janssen, Chiesi, Tillots, Kern, Amgen, Gebro and Cellgene. Alberto Malesci has received consultancies from DiaSorin. Roberta Pollastro and Fabio Moretti are employees of Quantyca and have no conflicts of interests to disclose.

Supporting Information S1

Click here for additional data file.