Comparison of the Mayo Endoscopy Score and the Ulcerative Colitis Endoscopy Index of Severity and the Ulcerative Colitis Colonoscopy Index of Severity.

Background and study aims: Endoscopy plays an essential role in managing patients with ulcerative colitis (UC), as it allows us to visualize and assess the severity of the disease. As such assessments are not always objective, different scores have been devised to standardize the findings. The main aim of this study was to assess the interobserver variability between the Mayo Endoscopy Score (MES), Ulcerative Colitis Endoscopy Index of Severity (UCEIS) and Ulcerative Colitis Colonoscopy Index of Severity (UCCIS) analyzing the severity of the endoscopic lesions in patients with ulcerative colitis. Patients and methods: This was a single-cohort observational study in which a colonoscopy was carried out on patients with UC, as normal clinical practice, and a video was recorded. The results from the video were classified according to the MES, UCEIS and UCCIS by three endoscopic specialists independently, and they were compared to each other. The Mayo Endoscopy Score (MES) was used to assess the clinical situation of the patient. The therapeutic impact was analyzed after colonoscopy was carried out.
Results: Sixty-seven patients were included in the study. The average age was 51 (SD ± 16.7) and the average MES was 3.07 (SD ± 2.54). The weighted Kappa index between endoscopists A and B for the MES was 0.8; between A and C 0.52; and between B and C 0.49. The intraclass correlation coefficient for UCEIS was 0.92 among the three endoscopists (CI 95 %: 0.83-0.96) and 0.96 for UCCIS among the three endoscopists (CI 95 % 0.94-0.97). A change in treatment for 34.3 % of the patients was implemented on seeing the results of the colonoscopy. Conclusions: There was an adequate, but not perfect, correlation between the different endoscopists for MES, UCEIS, UCCIS. This was higher with the last two scores. Thus, there is still some subjectivity to be minimized through special training, on assessing the seriousness of the endoscopic lesions in patients with UC. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commecial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Introduction

Endoscopy is an essential tool for the management of ulcerative colitis (UC) due to the fact that it allows us to carry out a visual assessment of the severity of the disease, as well as assessing the efficiency of treatment 1 2 . Today, this is of great importance, as endoscopic improvement and complete healing of the previously observed lesions represent the main aims for most patients. However, it is not easy to make a correct objective assessment of the severity of lesions on endoscopy. To improve assessment and homogenize endoscopic findings, different scores have evolved.

Despite the fact that there are multiple endoscopic indices described in the literature, the Mayo Endoscopy Score (MES) is the most used endoscopic index due to its simplicity. Nonetheless, two other indices have been developed over the last few years, and are being used in recent studies: the Ulcerative Colitis Endoscopy Index of Severity (UCEIS) and the Ulcerative Colitis Colonoscopy Index of Severity (UCCIS).

The MES was developed in 1987, by Schroeder et al 3 , during a controlled trial with slow-release oral mesalazine placebo for the treatment of active UC, in which a proctosigmoidoscopy was used to observe colonic mucosa and its improvement. This score assesses the vascular pattern, the reliability and the presence of erosions, scored from 0 to 3 in a simple way 4 . It has never been validated in any study, but through use in habitual clinical practice 5 . Other studies were developed afterwards to analyze the interobserver correlation, which varies from 0.45 to 0.75, as reported by Vashist et al 6 .

In 2012, the UCEIS index was created by Travis et al 7 as a tool to accurately predict the general assessment of the endoscopic severity in UC. This score is a scale of nine points (0–8) which assesses the vascular pattern, presence of bleeding and presence of erosions/ulcerations, each one having different, well-defined levels of severity. This index has been validated with an interobserver Kappa correlation of 0.50 8 . What is more, this index has been shown to be of great prognostic use in serious flare-ups of UC, in such a way that a UCEIS score ≥ 7 is correlated with a high likelihood of rescue treatment being administered (infliximab or cyclosporine) 9 .

Samuel et al 10 developed the UCCIS in 2013. This index ranges from 0 to 13. Its score is calculated by means of a formula that takes into account vascular pattern, granularity, ulcers, and presence of bleeding. It shows a good to excellent interobserver correlation and a good correlation with the index of clinical activity, the SCCAI (Simple Clinical Colitis Activity Index). This index is used to assess UC activity by means of clinical parameters, (excluding biochemical and endoscopic parameters) and the definitive remission of the patient (defined as a correlation coefficient of 0.61 to 0.80) 10 . However, the viability and the simplicity of use of UCCIS has not yet been assessed.

A study demonstrating a good correlation between UCEIS and MES (Kappa index 0.713, P  < 0.001) has recently been published 11 . Nonetheless, in the study by Ikeya et al 5 , it was observed that the UCEIS detected subtle changes in the mucosa better than MES, and that those changes correlated with survival. Also, in the study by Xie et al 12 , it was found that UCEIS was superior to MES concerning whether patients needed a colectomy after a severe outbreak of UC, presenting a ROC area of 0.85, with a sensitivity of 60.3 % and a specificity of 85.5 %; the ROC curve for MES was 0.65.

Of all the indices, the MES is the most used endoscopic index, both in normal clinical practice and in clinical trials, due to its simplicity. However, there are few studies comparing interobserver concordance within this index, let alone comparing the three indices with each other. This study was carried out to assess the interobserver concordance of these endoscopic indices and the impact their results have in clinical practice.

Patients and methods

The objective of our study was to analyze the interobserver concordance in endoscopic assessment of the degree of UC activity among three endoscopists according to three endoscopic scores (MES, UCEIS and UCCIS) and its impact in clinical practice.

This was a single-cohor,t observational, comparative study of patients diagnosed with UC who had undergone a colonoscopy, according to normal clinical practice, at the University Virgen Macarena Hospital (HUVM). These colonoscopies were carried out by any of the endoscopists at the hospital and a video was recorded anonymously and then seen by endoscopists A, B and C. Only one of them specialized in inflammatory bowel disease (A). All of them were familiar with MES and none use UCEIS and UCCIS in clinical practice.

The illness was classified according to the MES, UCEIS, and UCCIS by three different expert endoscopists. An expert endoscopist was considered as having more than 15 years of experience and/or having carried out more than 10,000 colonoscopies.

The video recordings were carried out using the Endobase program on withdrawing the colonoscopy, from the cecum to the rectum, or at least the most affected section, for no less than 60 seconds.

Inclusion criteria were age ≥ 18, previous diagnosis of UC, follow-up at the Inflammatory Bowel Disease section at HUVM, and a good-quality video of the colonoscopy, lasting at least 60 seconds and with an acceptable bowel preparation (Boston Bowel Preparation Score [BPPS] ≥ 4). Exclusion criteria were pregnancy, New York Heart Association > 2, severe chronic obstructive pulmonary disease, refusal to sign the form of consent for the colonoscopy, inadequate cathartic preparation (BPPS < 4 points), impenetrable colonic stenosis in sigma, suspicion of toxic megacolon, and an extensive intestinal resection (subtotal colectomy).

Patients with UC were included beginning in April 2019. A prospective follow-up was carried out to analyze their clinical evolution. Clinical data and data on the rest of the variables were gathered from the electronic digitized clinical histories in the DIRAYA program of the Andalusian Public Health System (SSPA) and tabulated according to the Mayo Clinic. The analysis was performed carried within an interval of ± 7 days after colonoscopy, and at 6 and 12 months.

Information on patient treatment when colonoscopy was carried out was gathered from the electronic digitized clinical histories in the DIRAYA program. Treatment continuation or change was assessed by the gastroenterologist in charge of the patient, according to the colonoscopy report. The changes in treatment were correlated with the clinical evolution of the patients at 6 and 12 months. The endoscopists who evaluated the videos, (A, B and C) had no prior knowledge of either the clinical situation or the treatment of the patients.

Data analysis

For the descriptive analysis, the following values were calculated: absolute frequency (N), relative frequency (%), average values, typical deviation (T.D.), minimum, maximum, and 25 %, 50 %, and 75 %. For the inference analysis, the confidence level was considered to be of 95 %, thus, the experimental P value was compared to the level of significance of 5 %.

To analyze the qualitative variables, a chi-squared test was used. To carry out the analysis between the categorical variable and another quantitative variable, it was necessary to know the most adequate type of test to be used according to the data. Normality tests were done by means of the Shapiro-Wilk or Kolmogorv-Smirnov test.

The tests carried out in the study were as follows. A Mann-Whitney U test was used for independent samples to analyze the change in treatment and its correlation with the endoscopic indices. The Friedman two-dimensional variance analysis was used to assess the evolution of patients according to the MCS.

Also, to indicate the intensity and the tendency in the relationship between two quantitative variables, a co-relational test was carried out. Pearson coefficient correlation analysis or the Spearman coefficient correlation analysis was used, depending on the behavior of those variables.

Finally, to evaluate concordance among the different measurements of the endoscopists, both Kappa and weighted Kappa scores were calculated, as well as the intraclass correlation coefficient.

This study was approved by the Research Ethics Board of the Virgen Macarena Hospital.

Results

There were 74 patients included in the study, three of whom were excluded, due to the fact that they did not have a histological diagnosis; another three, due to a failure in fulfilling the criteria of inclusion, as well as having Crohn’s disease, and another due to the fact that the video was invalid ( Fig. 1 ). Therefore, the final sample was of 67 patients, whose demographic characteristics can be found in Table 1 .

Fig. 1 

Flow diagram.

Demographic characteristics.

Demographic characteristics	N = 67 (%)
Sex
Female	31 (46.3 %)
Male	36 (53.7 %)
Mean age (DE)	51.2 (16.7)
Tobacco
Yes	16 (23.9 %)
No	48 (71.6 %)
Former smoking	3 (4.5 %)
Extent
Ulcerative proctitis	26 (38.8 %)
Left-sided UC	30 (44.8 %)
Extensive UC	11 (16.4 %)
Mean Mayo Clinic index (DE)	3.07 (2.54)
Treatment
Corticosteroid	1 (1.5 %)
Mesalazine	31 (46.3 %)
Corticosteroids + mesalazine	10 (14.9 %)
Mesalazine + azathioprine	12 (17.9 %)
Mesalazine + biological	7 (10.4 %)
Mesalazine + azathioprine + biologic treatment	6 (8.9 %)

DE, disease extent; UC, ulcerative colitis.

Interobserver correlation of MES, UCEIS and UCCIS

The weighted Kappa index between endoscopists A and B for MES was 0.8 (good); between A and C 0.52 (acceptable) and between B and C 0.49 (acceptable). The relationship between the index of endoscopists A and B according to the Spearman correlation coefficient was 0.88; between A and C 0.85 and between B and C 0.80. In Fig. 2 , the distribution of the MES results according to the different endoscopists is shown. If the index of central values (1,2) was compared to extreme values (0,3) a Kappa index of 0.69 was reached between endoscopists A and B, 0.24 between endoscopists A and C, and 0.23 between endoscopists B and C.

Fig. 2

 Distribution of MES according to the different endoscopists.

For the UCEIS, the interclass correlation coefficient of average values was 0.92 among the three endoscopists (CI 95 % 0.83–0,96). The interclass correlation coefficient between endoscopists A and B was 0.94 (excellent); between A and C 0.89 (excellent); and between B and C 0.92 (excellent). The relationship between the index of endoscopists A and B according to Spearman’s correlation coefficient was 0.87; between A and C 0.82 and between B and C 0.87. In Fig. 3 the distribution of UCEIS values according to the different endoscopists is observed. If the values were divided into two subgroups, the extreme values (UCEIS 0, 1 and 8) and the central values (2, 3, 4, 5, 6 and 7) of the index, a weighted Kappa of 0.71 was obtained between endoscopists A and B; 0.39 between A and C and 0.45 between endoscopists B and C.

Fig. 3

 Distribution of the Ulcerative Colitis Endoscopy Index of Severity according to the different endoscopists.

For the UCCIS an interclass correlation coefficient of 0.96 between the three endoscopists was reported, (CI 95 % 0.94–0.97) and on pairing the endoscopists the result was the following: 0.99 between endoscopists A and B; 0.91 between A and C and 0.91 between endoscopists B and C. The relationship between endoscopists A and B according to Spearman’s correlation coefficient was 0.97; 0.85 between A and C and 0.86 between B and C. In Fig. 4 the distribution of UCCIS values according to the different endoscopists is shown. Comparing the central values (2, 3, 4, 5, 6, 7, 8, 9) from the extreme values (0, 1, 10) we obtained a Kappa index of 0.85 between endoscopists A and B; 0.36 between A and C and 0.34 between endoscopists B and C.

Fig. 4

 Distribution of Ulcerative Colitis Colonoscopy Index of Severity according to the different endoscopists.

Correlation of the endoscopic scores with the clinical situation of the patient and changes in treatment

The mean MCS in the patients on carrying out the colonoscopy was 3.1 (SD ± 2.54) with an average of 3 (CI 1–5), at 6 months it was 1.47 (SD ± 1.80) 1 (CI 0–3) ( P  = 0.008) and at 12 months it was 0.94 (SD ± 1.27) 1 (CI 0–1) ( P  = 0.003). The correlation between the three endoscopic indices and the patients’ clinical situation (according to the MCS) is found in Table 2 .

Correlation between the endoscopic index and the Mayo Clinic Score.

	Spearman correlation coefficient
	Endoscopist AMayo Clinic	Endoscopist BMayo Clinic	Endoscopist CMayo Clinic
MES	0.53	0.53	0.60
UCEIS	0.56	0.65	0.63
UCCIS	0.66	0.66	0.61

MES, Mayo Endoscopy Score; UCEIS, Ulcerative Colitis Endoscopy Index of Severity; UCCIS, Ulcerative Colitis Colonoscopy Index of Severity.

In 34.3 % of the patients, the result of the colonoscopy led to changes in treatment, of whom 87 % underwent an intensified treatment, and in 13 %, the treatment was withdrawn or de-intensified. The median basal MCS in those patients who had a change in their treatment after colonoscopy was 4.63 (SD ± 2,36), with an average of 4.5 (CI 3–6.75). While there were 1.59 patients (SD ± 2.01), an average of only one patient (CI 0–3) had no change in treatment ( P  < 0.001). An MCS ≥ 2 was present in 95.8 % of patients whose treatment was changed after colonoscopy ( P  < 0.001). In the group of patients who underwent a change in their treatment, the MCS decreased from 4.5 at baseline to 3 points at 6 months ( P  = 0.075) and 1 point at 12 months ( P  = 0.007). In those patients who had no change in treatment, the MCS was on average 1 at baseline, 0 at 6 months, and 0 at 12 months (the difference between the baseline and that at 12 months was not statistically significant, P  = 0.119).

The relationship between the change in treatment after colonoscopy and the findings according to the different endoscopists was studied. These results are shown in Table 3 .

Relationship between the change in treatment after carrying out colonoscopy and findings per endoscopist according to the MES, UCEIS and UCCIS indices.

	Endoscopist A	Endoscopist B	Endoscopist C
MES 3. Change of treatment	66.7 % ( P  = 0.008)	75.0 % ( P  = 0.002)	92.9 % ( P  = 0.018)
MES 0. Maintenance of treatment	88.2 % ( P  = 0.008)	93.3 % ( P  = 0.002)	55.6 % ( P  = 0.018)
UCEIS  ≥  3. Change of treatment	58.3 % ( P  = 0.012)	62.5 % ( P  = 0.002)	79.2 % ( P  = 0.014)
UCEIS < 3. Maintenance of treatment	75.0 % ( P  = 0.012)	78.1 % ( P  = 0.002)	53.1 % ( P  = 0.014)
UCCIS  ≥  3. Change of treatment	79.2 % ( P  = 0.001)	75.0 % ( P  < 0.001)	100 % ( P  < 0.001)
UCCIS < 3. Maintenance of treatment	65.6 % ( P  = 0.001)	75.0 % ( P  < 0.001)	40.6 % ( P  < 0.001)

MES, Mayo Endoscopy Score; UCEIS, Ulcerative Colitis Endoscopy Index of Severity; UCCIS, Ulcerative Colitis Colonoscopy Index of Severity.

Discussion

To diagnose UC, it is essential to obtain clinical, biochemical, endoscopic, and pathologic data. Thus, colonoscopy plays an essential role in making a differential diagnosis in patients with UC, and it allows us to visualize and assess the severity of the disease, help plan therapeutic management and estimate the disease’s prognosis. However, it is not easy to make a correct objective assessment of the mucosa, let alone have all of the endoscopists grade the severity of the illness in the same way.

In a recent review by Vashist et al 6 , the different studies assessing the interobserver correlation of the indices were analyzed. The Kappa index was reported to vary on MES from 0.45 to 0.75 (acceptable to good). An interobserver correlation with a Kappa index of 0.53 (CI 95 %: 0.47–0.56) for expert endoscopists, and 0.71 (CI 95 %:0.67–0.76) for non-expert endoscopists was observed 13 . In our study, a good correlation in MES was found, with a weighted Kappa index of between 0.8 (good) and 0.49 (acceptable), as reported in other papers.

On the other hand, dividing variables between extreme values and central values, it has been observed that this correlation decreases to 0.23 to 0.69. Similar findings were reported in the study by Fernándes et al 14 , in which the correlation of MES was 0.47 (CI 95 % 0.41–0.54), yet when only Mayo 0 was analyzed, this correlation increased to 0.89 (CI 95 % 0.73–1). This points to the difficulty of this index to catalog endoscopic findings within the central subgroup (1 and 2) and the extremes (0 and 3). Indeed, this is in line with the study by Ikeya et al 5 , which observed that the UCEIS was better than MES in detecting subtle changes in the mucosa, probably due to the fine line between the central categories.

In the study by Travis et al 7 , every item assessed in the UCEIS index was analyzed. The weighted Kappa index for the reliability value was 0.3 (not acceptable), while it was 0.45 (acceptable) for the values of erosions and ulcers. Later, in another study by the same author, the interobserver correlation for the same index was analyzed. The weighted Kappa index score was 0.47 (acceptable) (CI 95 % = 0.46, 0.49) and 0.47 (acceptable) (CI 95 % = 0.44, 0.50) for blinded and non-blinded readers, respectively 15 . In our study, the interobserver variability of the index was analyzed globally, and a good correlation was found (0.92) with good scores for Rho Spearman (between 0.82 and 0.87). This indicates that there was good concordance among the three endoscopists regarding assessing the mucosa of the patients with UC. The improvement in our results is probably due to the fact that a general analysis was carried out, as opposed to subgroups within said index, and its different subsections (mucosa assessment, reliability, and erosions/ulcers), on the one hand, and, on the other, the number of patients included in each study (41 in the study by Travis versus 67 in ours).

Concerning the general analysis of the correlation of the UCEIS index and its results, when we analyzed the interobserver correlation differentiating central values (2, 3, 4, 5, 6, 7) to extreme values (0, 1, 8), the results dropped quite significantly, with a Kappa index of 0.71 between endoscopists A and B (as the best result) and 0.39 between endoscopists A and C.

In the review carried out by Vashist et al 6 , the correlation for the UCCIS index was also analyzed, with results taken from other studies ranging from 0.56 to 0.88. Our result was somewhat higher, with an interclass correlation of 0.96 (CI 95 % 0.94–0.97). The fact that this is an index used less in clinical practice probably had an effect on this result, as indeed did the fact that the endoscopists carried out the assessment using this index after the parameters taken into account within the index were explained.

In the 2011 study by Daperno et al 16 , in which 171 gastroenterologists assessed five different videos of colonoscopies before and after receiving specific training for the MES, it was reported that the correlation improved after the training from 0.45 to 0.71. The gastroenterologist’s assessment will have an effect on the decisions that will be taken, as indeed we have seen in our study, in which 34.3 % of the patients underwent a change in their treatment after the specialist saw the results. Thus, it is extremely important that the gastroenterologists who assess this type of patient should be properly trained.

Regarding the relationship that exists among the endoscopic indices and the clinical situation of a patient, according to our results, all three indices (MES, UCEIS and UCCIS) have a good correlation, which ranged from 0.53 to 0.65. This is similar to findings reported in literature, such as the study by Fluxá 17 which showed a moderate correlation for MES with a patient’s clinical situation. The study by Samuel et al is another example for UCCIS 10 . The index correlation was carried out using the clinical scale SCCAI and the CAI scale or the Rachmilewitz score, with results similar to ours (0.62; P  < 0.0001 and 0.52; P  < 0.001, respectively). However, in the study by Travis that was published in 2015, a higher correlation between the index and the symptomatology of a patient was reported (with an average of 0.899 and 0.933 in the blinded and non-blinded groups, respectively) 15 .

In 34.3 % of patients in our cohort, the result of colonoscopy led to changes in treatment, showing a drop by 4 points throughout the 12-month-follow-up in the MCS (from 5 at base level to 1 at 12 months), which was statistically significant ( P  = 0.007). Said change in treatment was established after seeing the endoscopic result, which in itself reflects the importance of the examination and the impact it may have on a patient’s symptoms and improvement. Likewise, patients who did not undergo any change in their treatment, had an average baseline MCS of 1 and 0 at 6 and 12 months ( P  = 0.03). Also, according to the different endoscopists, more than 70 % to 80 % of the patients presented an MES of 0, a UCEIS < 3 and a UCCIS < 3, thus showing the clinical stability after 1 year of patients who had no change in treatment after the result of the colonoscopy.

Therefore, the importance of not only carrying out a colonoscopy, but also its correct assessment, is clearly evident. It represents a detailed study which allows us to grade the illness, carry out a more adequate and objective follow-up, as well as decide on therapeutic modifications that help improve and maintain the patient’s condition. In this way, as demonstrated in studies such as that by Barreiro et al, mucosal healing improves the evolution of the illness 18 .

As far as we know, our study is the first to directly compare the three endoscopic indices used to assess UC activity. Our results allow us to demonstrate that the UCEIS and the UCCIS indices are superior to MES, regarding interobserver correlation. Thus, although their interpretation could be considered as more complex, we consider they should be used more in clinical practice.

Our study, however, does have some limitations worthy of analysis. On the one hand, not all the videos showed a complete examination. This could indeed lead to an underestimation of the stage of the illness. Nonetheless, the three endoscopists did receive the same video, thus, the assessment was carried out on the same images of the mucosa. Moreover, the quality of the images on screen live is better than on the video, which could indeed influence their assessment. On the other hand, not all of the patients had good or excellent cathartic preparation with a BPPS of 8 to 9. This could impede better visualization of the colonic mucosa and certain patterns, for example, the vascular pattern. We consider that the fact that all of the patients were from the same center should not be taken as a limitation; however, the fact that all of the endoscopists were from the same center could indeed be considered as such or as a methodological bias within the study. A histological correlation was not carried out. Indeed, it is true to say that in some recent studies, the histological healing of the mucosa has been the aim. Up until now, however, this has not been proven to be efficient. What is more, there is no standardized point system for this. Finally, calprotectine and its correlation with the endoscopic findings were not investigated. This certainly would have been interesting; however, at the beginning of the study, we did not have access to said test in our center, therefore, we were unable to carry it out.

Conclusion

In conclusion, we consider that there is an adequate, but not perfect, correlation among the different endoscopists for MES, UCEIS and UCCIS (the latter being the best). These indices have an acceptable correlation with the clinical situation of a patient and their results affect therapeutic decisions. Indeed, an improvement was reported in patients who were analyzed at 12 months.