Performance report cards increase adenoma detection rate.

BACKGROUND AND STUDY AIMS: Adenoma detection rate (ADR) is an important measure of colonoscopy quality, as are polyp, advanced ADR, and adenocarcinoma detection rates. We investigated whether performance report cards improved these outcome measures. PATIENTS AND METHODS: Endoscopists were given report cards comparing their detection rates to the institutional mean on an annual basis. Detection rates were evaluated at baseline, 1 year after report cards (Year 1), and 2 years after report cards (Year 2). Endoscopists were unaware of the study and received no other interventions. The primary outcome was ADR and secondary outcomes were polyp detection rate (PDR), advanced ADR, and adenocarcinoma detection rate. Multivariate regression was performed to adjust for temporal trends in patient, endoscopists, and procedural factors.
RESULTS: Seventeen physicians performed 3,118 screening colonoscopies in patients with positive FOBT or family history of colon cancer. The ADR increased from 34.5 % (baseline) to 39.4 % (Year 1) and 41.2 % (Year 2) ( P  = 0.0037). The PDR increased from 45 % (baseline) to 48.8 % (Year 1) and 51.8 % (Year 2) ( P  = 0.011). There was no significant improvement in advanced ADR or adenocarcinoma detection rates. On multivariate analysis, the ADR increased by 22 % in Year 1 ( P  = 0.03) and 30 % in Year 2 ( P  = 0.008). Among physicians with a baseline ADR < 25 %, improvement in ADR was even greater, increasing 2.2 times by the end of the study ( P  = 0.004). Improvements in ADR were not correlated with specialty although gastroenterologists were 52 % more likely to find an adenoma than general surgeons.
CONCLUSIONS: Annual performance report cards increased adenoma detection rates, especially among physicians with low ADR < 25 %.

Introduction

Colon cancer is the second leading cause of cancer related deaths 1 . The sequence between adenoma to adenocarcinoma takes on average 10 years and can be prevented by polypectomy 2 . Numerous observational studies have reported reduced colorectal cancer mortality with the performance of colonoscopies and randomized clinical trials are underway 3 4 5 6 7 8 9 10 11 . Currently, colonoscopy is one of the major forms of colon cancer screening along with fecal testing 12 .

Quality assurance is an integral part of an effective screening colonoscopy program. Adenoma detection rate (ADR) has emerged as one of the most important quality indicators. ADRs less than 20 % have been associated with development of interval colon cancer after a screening colonoscopy 13 14 . As a result, recent guidelines recommend an ADR ≥ 25 % 15 . However, there is considerable variation in the ADR between endoscopists 16 17 . Numerous interventions to improve the ADR, such as increased withdrawal time, enhanced inspection techniques, and/or educational interventions have been largely ineffective 18 . In this study, we sought to determine the effect of performance report cards on the ADR over a 3-year period.

Patients and methods

Study design

The study was conducted at St. Joseph’s Health Care, an academic hospital affiliated with Western University (London, Ontario, Canada), between April 1, 2012 and Mar 31, 2015 as a part of a colonoscopy quality assurance initiative through the ColonCancerCheck™ (CCC) program of the Southwestern Ontario Regional Cancer Program and Cancer Care Ontario. The CCC was the first programmatic provincial colon cancer screening program in Canada, whereby average risk individuals aged 50 to 75 are encouraged to have biennial fecal occult blood testing (FOBT). All tests are processed at CCC approved labs using Hemoccult II Sensa (Beckman Coulter, Mississauga, Canada). Those with positive tests are then referred for colonoscopy. Patients with a family history of colon cancer in a first-degree relative are considered “high risk” and referred directly for colonoscopy. Thus, patients included in this study were those with either positive FOBT or a family history of colon cancer in a first-degree relative having their first colonoscopy. Colonoscopies completed on symptomatic patients, average risk screening, or surveillance for colorectal neoplasm/polyps were excluded. All endoscopists involved in the program were board-certified gastroenterologists or general surgeons who performed at least 200 colonoscopies each year for any indication. Residents and fellows were not involved in any of the procedures. The protocol was approved by the university’s Research Ethics Board.

Beginning on April 1, 2012, physicians were required to complete a colonoscopy reporting tool for all colonoscopies documenting the procedural indication, bowel preparation quality, cecal intubation, gross findings including detection of polyp(s), interventions completed, and immediate adverse events. Bowel preparation quality was rated using an ordinal scale as follows: good (mucosa seen throughout), fair (liquid contents, exam adequate), and poor (solid contents, exam compromised). The decision to use a simple ordinal scale was made by Cancer Care Ontario due to the lack of a standardized bowel preparation score used throughout the province. The tool was completed immediately after each procedure and participation was mandatory. In patients who were identified as having at least 1 polyp removed, histological information was manually reviewed to determine the polyp type. Because the quality assurance program was centered on programmatic colonoscopy screening, and due to the very high number of annual colonoscopies completed at the center, histology was not reviewed for other indications.

The study was divided into 3 phases. Between April 1, 2012 and March 31, 2013 (Baseline), physicians were not aware that the data from the colonoscopy reporting tool would be used to generate report cards. Personalized report cards were issued in April 2013 documenting a physician’s cecal intubation rate, preparation quality, ADR, polyp detection rate (PDR), advanced ADR, adenocarcinoma detection rate, and perforation rate compared to the institutional mean. A second report card was issued in April 2014. The period between April 1, 2013 and March 31, 2014 defined Year 1 and the period between April 1, 2014 and March 31, 2015 defined Year 2. Report cards were confidential and not shared with anyone other than the individual physician. There were no interventions other than the report cards. In addition, physicians were not aware of the study during the data collection period.

Outcome definitions

ADR was chosen as the primary outcome and defined as the proportion of patients undergoing colonoscopy with 1 or more adenomas confirmed histologically. Secondary outcomes included PDR, advanced ADR, and adenocarcinoma detection rate. PDR was defined as the proportion of patients undergoing colonoscopy with 1 or more polyps of any histology. An advanced adenoma was defined as having any of the following: ≥ 1 cm in diameter, villous histology, or high-grade dysplasia.

Statistical analysis

Individual patient level data were collected prospectively for each procedure. Crude analyses to compare the ADR, PDR, advanced ADR, and adenocarcinoma detection rates between baseline and Year 1 and baseline and Year 2 were performed using analysis of variance and pairwise comparisons. A Bonferonni correction was deemed unnecessary because the comparisons were only between 3 groups 19 . Multivariate logistic regression using generalized estimating equations was performed to adjust for patient (indication, age, gender) and procedural (physician specialty, bowel preparation quality, and cecal intubation) factors, which were defined a priori based on subject knowledge rather than empiric methods 15 16 20 . Given that ADR may differ for colonoscopies performed for positive FOBT and family history, we tested for effect modification by procedure indication. Furthermore, we examined for effect modification by specialty (gastroenterology versus general surgery), and baseline ADR (< 25 % versus ≥ 25) using interaction terms. The cut off for baseline ADR was chosen to be 25 % based on current targets recommended by recent guidelines 15 . A 2-sided P value < 0.05 was considered significant. All statistical analyses were performed with Stata 14.0.

Results

Between April 1, 2012 and March 31, 2015, 17 physicians (11 gastroenterologists, 6 general surgeons) performed 3,118 colonoscopies on patients with a family history of colorectal cancer (76.6 %) or a positive FOBT (23.4 %) ( Table 1 ). The mean (SD) patient age was 58.6 (10.5) and 59.8 % were female. Bowel preparation quality was rated as good, fair, and poor in 89.3 %, 8.6 %, and 2.1 %, respectively. Cecal intubation was successful in 96.6 % of patients.

Characteristics of 3,118 patients undergoing colonoscopies during the study period.

Age-mean (SD)	58.6 (10.5)
Female-no. (%)	1864 (59.8 %)
Indication-no.(%)
Family history of colon cancer	2388 (76.6 %)
Positive FOBT	730 (23.4 %)
Bowel preparation quality
Good	2,784 (89.3 %)
Fair	269 (8.6 %)
Poor	65 (2.1 %)
Cecal intubation
Yes	3,012 (96.6 %)
No	94 (3.0 %)
Not applicable 1	12 (0.4 %)

Includes cases such as obstructing tumor and altered surgical anatomy

On crude analysis, ADR improved from a baseline of 34.5 % to 39.4 % in Year 1 ( P  = 0.0152) and to 41.2 % in Year 2 ( P  = 0.0016) ( Fig. 1 and Table 2 ). The increase in ADR between Year 1 and Year 2 was not statistically significant ( P  = 0.33). The PDR increased from a baseline of 45 % to 48.8 % in Year 1 (0.0684) and to 51.8 % in Year 2 ( P  = 0.0032). The advanced ADR and adenocarcinoma rates improved each year but did not reach statistical significance.

Fig. 1 

Detection rates by year.

Crude analysis of adenoma, polyp, advanced adenoma, and adenocarcinoma detection rate by year.

	Detection rate-no. (%)			P value
	Baseline (n = 1,133)	Year 1 (n = 1,172)	Year 2 (n = 813)	Global	Year 1 1	Year 2 1
Adenoma detection rate	391 (34.5 %)	462 (39.4 %)	338 (41.2 %)	0.0037	0.0152	0.0016
Polyp detection rate	510 (45.0 %)	572 (48.8 %)	421 (51.8 %)	0.0113	0.0684	0.0032
Advanced adenoma detection rate 2	78 (6.9 %)	95 (8.1 %)	76 (9.4 %)	0.14	0.28	0.05
Adenocarcinoma detection rate	8 (0.7 %)	10 (0.9 %)	10 (1.2 %)	0.4721	0.7082	0.2271

Compared to baseline

Defined as > 1 cm in size, presence of villous component, or high grade dysplasia

On multivariate analysis controlling for patient age and gender, physician specialty, indication, bowel preparation quality, and cecal intubation, ADR increased by 22 % in Year 1 (OR 1.22, 95 % CI 1.02 – 1.45, p = 0.03) and 30 % in Year 2 (OR 1.30, 95 % CI 1.07 – 1.58, P  = 0.008) compared to baseline ( Table 3 ) (see Supplement for model details). The effect of the report card intervention on the ADR did not differ based on the indication for colonoscopy (interaction term P value > 0.05). However, there was evidence of effect modification by baseline ADR when tested with an interaction term, indicating the effect of report cards differed based on baseline ADR (interaction term P value = 0.03). Among physicians with a baseline ADR < 25 %, ADR increased 2.21 times after Year 1 ( P  = 0.003) and 2.17 times ( P  = 0.004) after Year 2. Among physicians with a baseline ADR ≥ 25 %, ADR increased by 18 % ( P  = 0.10) after Year 1 and 33 % ( P  = 0.01) after Year 2 ( Table 4 ) (see Supplement for model details).

Multivariate analysis 1 of adenoma, polyp, advanced adenoma, and adenocarcinoma detection rates compared to baseline.

	Year 1 OR (95 % CI)	P value	Year 2 OR (95 % CI)	P value
Adenoma	1.22 (1.02 – 1.45)	0.03	1.30 (1.07 – 1.58)	0.008
Polyp	1.14 (0.96 – 1.35)	0.13	1.28 (1.06 – 1.54)	0.01
Advanced adenoma 2	1.12 (0.81 – 1.54)	0.50	1.27 (0.90 – 1.78)	0.18
Adenocarcinoma	1.30 (0.48 – 3.53)	0.61	2.04 (0.75 – 5.58)	0.16

Adjusted for endoscopist specialty, patient age, gender, procedural indication, bowel preparation quality, and cecal intubation

Includes adenoma with any of the following features: > 1 cm in size, villous component, or high-grade dysplasia

Multivariate analysis of factors influencing adenoma detection rate.

	Odds Ratio (95 % CI)	p-value
Year 1 1	1.22 (1.02 – 1.45)	0.03
Year 2 1	1.30 (1.07 – 1.58)	0.008
GI specialty 2	1.52 (1.28 – 1.81)	< 0.001
Age	1.04 (1.03 – 1.05)	< 0.001
Male gender	1.96 (1.67 – 2.27)	< 0.001
Positive FOBT	1.20 (0.99 – 1.44)	0.05
Good bowel preparation 3	4.32 (1.96 – 9.51)	< 0.001
Fair bowel preparation 3	4.54 (2.14 – 9.66)	< 0.001
Cecal intubation	6.78 (3.30 – 13.91)	< 0.001

Compared to baseline

Compared to general surgery

Compared to poor bowel preparation

Improvement in adenoma detection rate by baseline adenoma detection rate 1 .

	Crude analysis Adenoma detection rate (%)						Multivariate analysis 1 OR (95 % CI) for adenoma detection compared to baseline
Baseline ADR	Baseline	Year 1	Year 2	Global p-value	P value 2 (Year 1)	P value 2 (Year 2)	Year 1	Year 2	P value (Year 1)	P value (Year 2)
< 25 %	28/176 (15.9 %)	67/254 (26.4 %)	60/210 (28.6 %)	0.009	0.01	0.004	2.21 (1.32 – 3.71)	2.17 (1.28 – 3.68)	0.003	0.004
≥ 25 %	363/957 (37.9 %)	395/918 (43.0 %)	278/603 (46.1 %)	0.004	0.03	0.001	1.18 (0.97 – 1.43)	1.33 (1.08 – 1.66)	0.10	0.01

Adjusted for endoscopist specialty, patient age, gender, procedural indication, bowel preparation quality, and cecal intubation

Compared to baseline

The overall ADR was 40.6 % for gastroenterologists and 32.0 % for general surgeons ( P  < 0.0001) ( Table 5 ). Gastroenterologists were 52 % more likely to detect an adenoma than a general surgeon on multivariate analysis (OR 1.52, 95 % CI 1.28 – 1.81) (see Supplement for model details). However, the change in ADR in Year 1 and Year 2 did not differ significantly by specialty (interaction term P value > 0.05).

Crude adenoma detection rate by specialty.

	Adenoma detection rate (%)
	Gastroenterologist	General Surgeon	P value
Baseline	308/847 (36.4 %)	83/286 (29.0 %)	0.02
Year 1	350/842 (41.6 %)	112/330 (33.9 %)	0.02
Year 2	259/527 (45.3 %)	79/241 (32.8 %)	0.0009
Total	917/2,216 (40.6 %)	274/857 (32.0 %)	< 0.0001

Discussion

Reduction in colorectal cancer mortality from screening colonoscopy relies on identification and removal of precancerous polyps 21 . Accordingly, ADR has emerged as one of the most important quality metric in colonoscopy. In this study, we found that issuing annual performance report cards significantly improved ADR. On multivariate analysis, ADR improved by a striking 22 % in Year 1 and 30 % in Year 2 compared to baseline. The benefit of report cards depended on baseline ADR. Among those who were below recommended ADR at baseline (< 25 %), it improved 2.2 times by the end of Year 2. Accordingly, ADR in this group increased from 15.9 % at baseline to 28.6 % by the end of the study. Even among high performers who had a baseline ADR ≥ 25 %, ADR improved by 33 % after Year 2. Thus, our results support use of performance report cards to improve ADRs for all endoscopists.

In the absence of well-designed placebo-controlled, randomized clinical trials (RCTs), observational studies have been used to justify the effectiveness of performance report cards on improving ADR. Although our study is not the first to examine this intervention, it is the first to do so in a more rigorous manner. Past studies examining the effect of report cards on ADR or PDR either did not account for confounding variables 22 23 24 25 or controlled only for age and gender 26 . Given the strong association between variables such as patient age, gender, cecal intubation, and bowel preparation quality with ADR 15 16 20 , failure to adjust for these factors may lead to erroneous conclusions. As an example, improvements in bowel preparation quality over time may have explained improvements in ADR observed independent of the impact of report cards in prior uncontrolled studies. This is particularly true since bowel preparation itself was a target for quality improvement during the period during which the prior studies were conducted. In our protocol, we prospectively measured potential confounders immediately after each colonoscopy and adjusted for them in a multivariate analysis, leading to a more accurate assessment of the impact of report cards on ADR. After controlling for confounders, we still found a significant improvement in ADR. To our knowledge, our study provides the most rigorous evidence for the effectiveness of performance report cards for improving ADR to date.

A recent RCT comparing performance report cards to an intensive quality improvement intervention bears mentioning 27 . In that study, colonoscopy screening center leaders who did not meet a target 25 % ADR were randomized to receive either performance report cards or an intensive quality improvement initiative. This consisted of a 2-hour pretraining assessment, 2 days intensive hands-on training in the United Kingdom by expert colonoscopists, and further evaluation of the first 30 colonoscopies performed upon return to their home endoscopy center. Among endoscopists randomized to performance report cards, ADR increased by 2.3 %, which is lower than our observed 6.7 % increase. Although direct comparisons between studies are difficult, we postulate several hypotheses to explain the difference. First, their patient population consisted of average-risk screening and was thus inherently different from ours. Second, endoscopists targeted in their study were those who did not meet a benchmark ADR > 25 % and were mostly physicians in private practice. Third, the inherent study design was different as theirs was experimental and ours observational. Nonetheless, although the magnitudes of the effect estimates are different, it is reassuring that both studies demonstrated a significant improvement in ADR after the intervention.

In our study, there were no interventions other than performance report cards. Furthermore, report cards were confidential and not shared with anyone other than the endoscopists. There were no penalties, meetings with departmental chiefs, shaming, or other coercive actions to induce practice changes. Unlike other studies, the only motivation was the report card itself. We postulate that seeing one’s performance objectively compared to others provided the introspection and motivation required for change.

Other than ADR, PDR improved over time as well. By the end of the study, the PDR had increased by 28 % ( P  = 0.01). However, we chose to focus on ADR as our primary outcome since PDR is subject to gaming. This is of particular importance for an intervention such as ours because it is possible for unscrupulous endoscopists to remove small hyperplastic polyps from the distal colon to improve this metric. There was also a trend towards improvement in detection of advanced adenomas although that did not reach statistical significance, likely due to the low event rate. Similarly, there were no more than 10 adenocarcinomas each year, making it unfeasible to draw reasonable conclusions about the effects of report cards on that outcome. Lastly, we found that ADR was affected by physician specialty. Prior studies using administrative data to determine interval colorectal cancer rates have reported differences between gastroenterologists, general surgeons, internists, and family physicians, although the results have been inconsistent 28 29 30 31 . In our study, all endoscopists used the same equipment on the same patient population and gastroenterologists were 52 % more likely to detect an adenoma than were general surgeons. The reason for this discrepancy is beyond the scope of our study although differences in formal endoscopy training have been postulated previously 30 .

The main limitation of our study is the lack of randomized allocation of the study intervention. Because this was a quality assurance intervention, all physicians were issued report cards during the study. Without randomization, there is a risk for biased results due to confounding variables. To address this possibility, we measured and adjusted for known confounders of ADR as reported in the literature. Nonetheless, there is still a risk for residual confounding by unmeasured factors. However, because the criteria for CCC screening did not change during the study period, it is unlikely to explain our findings. As a result, we believe our study provides a more robust effect estimate of the impact of performance report cards on ADR than previous studies that did not adjust for confounding. In our study, we chose to sample high-risk patients who were undergoing colonoscopy due to a history of colon cancer in a first-degree relative or who had a positive FOBT. We selected these patients because prior studies only focused on the general screening population and identifying a benchmark ADR for high-risk patients was previously identified as an unmet research need 15 . Nonetheless, there is no biological reason to expect that our intervention would be ineffective in the general screening population although the magnitude of improvement may differ, as it would when applied to different physicians, endoscopy units, and geographic locations. Lastly, although we did not record details regarding the bowel preparation itself (ie. type, dose, usage of split dosing, etc), our analysis adjusted for the quality of bowel preparation at the time of the procedure to provide an unbiased estimate of the treatment effect.

Conclusion

In conclusion, annual performance report cards increased ADR. The magnitude of the benefit was greatest in those who were below the threshold of 25 % ADR although everyone benefited. Our results support institutional adoption of this inexpensive and non-invasive intervention to improve these quality metrics.

Multivariate analysis of factors influencing polyp detection rate.

	Odds Ratio (95 % CI)	p-value
Year 1 1	1.14 (0.96 – 1.35)	0.13
Year 2 1	1.28 (1.06 – 1.54)	0.01
GI specialty 2	1.56 (1.32 – 1.84)	< 0.001
Age	1.03 (1.02 – 1.04)	< 0.0014
Female gender	0.54 (0.47 – 0.63)	< 0.001
Positive FOBT	1.05 (0.87 – 1.26)	0.61
Good bowel preparation 3	3.35 (1.70 – 6.62)	< 0.001
Fair bowel preparation 3	3.58 (1.89 – 6.79)	< 0.001
Cecal intubation	5.61 (3.09 – 10.20)	< 0.001

Compared to baseline

Compared to general surgery

Compared to poor bowel preparation

Multivariate analysis of high risk adenoma detection rate.

	Odds Ratio (95 % CI)	p-value
Year 1 1	1.12 (0.81 – 1.54)	0.50
Year 2 1	1.27 (0.90 – 1.78)	0.18
GI specialty 2	1.30 (0.93 – 1.82)	0.13
Age	1.04 (1.02 – 1.05)	< 0.001
Female gender	0.55 (0.42 – 0.72)	< 0.001
Positive FOBT	3.42 (2.59 – 4.53)	< 0.001
Good bowel preparation 3	5.62 (0.72 – 43.97)	0.10
Fair bowel preparation 3	7.66 (1.03 – 57.13)	0.05
Cecal intubation	3.62 (1.10 – 11.89)	0.03

Compared to baseline

Compared to general surgery

Compared to poor bowel preparation

Multivariate analysis of adenocarcinoma detection rate

	Odds Ratio (95 % CI)	p-value
Year 1 1	1.30 (0.48 – 3.53)	0.61
Year 2 1	2.04 (0.75 – 5.58)	0.16
GI specialty 2	1.62 (0.47 – 5.60)	0.45
Age	1.08 (1.04 – 1.12)	< 0.001
Female gender	1.00 (0.46 – 2.22)	0.99
Positive FOBT	12.28 (4.07 – 37.05)	< 0.001
Good bowel preparation 3	–	–
Fair bowel preparation 3	–	–
Cecal intubation	0.33 (0.10 – 1.08)	0.07

Compared to baseline

Compared to general surgery

Compared to poor bowel preparation

Multivariate analysis of adenoma detection rate for physicians with baseline ADR < 25 %.

	Odds Ratio (95 % CI)	p-value
Year 1 1	2.21 (1.32 – 3.71)	0.003
Year 2 1	2.17 (1.28 – 3.68)	0.004
GI specialty 2	1.01 (0.67 – 1.52)	0.96
Age	1.04 (1.02 – 1.06)	< 0.001
Female gender	0.49 (0.33 – 0.73)	< 0.001
Positive FOBT	1.60 (1.00 – 2.55)	0.05
Good bowel preparation 3	–	–
Fair bowel preparation 3	–	–
Cecal intubation	5.47 (1.18 – 25.33)	0.03

Compared to baseline

Compared to general surgery

Compared to poor bowel preparation

Multivariate analysis of adenoma detection rate for physicians with baseline ADR ≥ 25 %.

	Odds Ratio (95 % CI)	p-value
Year 1 1	1.18 (0.97 – 1.43)	0.10
Year 2 1	1.33 (1.07 – 1.66)	0.01
GI specialty 2	1.42 (1.16 – 1.75)	0.001
Age	1.04 (1.03 – 1.05)	< 0.001
Female gender	0.50 (0.42 – 0.59)	< 0.001
Positive FOBT	1.17 (0.95 – 1.43)	0.14
Good bowel preparation 3	4.83 (2.16 – 10.83)	< 0.001
Fair bowel preparation 3	5.03 (2.34 – 10.82)	< 0.001
Cecal intubation	6.40 (2.81 – 14.60)	< 0.001

Compared to baseline

Compared to general surgery

Compared to poor bowel preparation