Patient comfort scores do not affect endoscopist behavior during colonoscopy, while trainee involvement has negative effects on patient comfort.

INTRODUCTION: Patient comfort is an important part of endoscopy and reflects procedure quality and endoscopist technique. Using the validated, Nurse Assisted Patient Comfort Score (NAPCOMS), this study aimed to determine whether the introduction of NAPCOMS would affect sedation use by endoscopists. PATIENTS AND METHODS: The study was conducted over 3 phases. Phase One and Two consisted of 8 weeks of endoscopist blinded and aware data collection, respectively. Data in Phase Three was collected over a 5-month period and scores fed back to individual endoscopists on a monthly basis.
RESULTS: NAPCOMS consists of 3 domains - pain, sedation, and global tolerability. Comparison of Phase One and Two, showed no significant differences in sedative use or NAPCOMS. Phase Three data showed a decline in fentanyl use between individual months ( P  = 0.035), but no change in overall NAPCOMS. Procedures involving trainees were found to use more midazolam ( P  = 0.01) and fentanyl ( P  = 0.01), have worse NAPCOMS scores, and resulted in longer procedure duration ( P  < 0.001). Data comparing gastroenterologists and general surgeons showed increased fentanyl use ( P  = 0.037), decreased midazolam use ( P  = 0.001), and more position changes ( P  = 0.002) among gastroenterologists.
CONCLUSIONS: The introduction of a patient comfort scoring system resulted in a decrease in fentanyl use, although with minimal clinical significance. Additional studies are required to determine the role of patient comfort scores in quality control in endoscopy. Procedures completed with trainees used more sedation, were longer, and had worse NAPCOMS scores, the implications of which, for teaching hospitals and training programs, will need to be further considered.

Introduction

Colonoscopy is used for evaluation, diagnosis and treatment of gastroenterological disorders. Colon cancer is the third most commonly diagnosed cancer in North America 1 2 . Fecal occult blood testing and colonoscopy remain 2 of the most widely used screening tools for colorectal cancer 3 4 .

There can be considerable variation in the quality of endoscopy and quality assurance in this field continues to gain importance in the eyes of administrators, endoscopists, and patients 5 . The American Society for Gastrointestinal Endoscopy identified 3 priority quality indicators for colonoscopy: adenoma detection rate (ADR), cecal intubation rate (CIR), and adherence to surveillance protocols 6 . A higher CIR is associated with increases in ADR 7 but a focus on CIR may result in endoscopists using more force to advance the instrument, with concomitant increased sedation requirements. An unintended consequence of this may be an increase in complications such as perforation. Numerous societies have proposed sedation as a quality indicator during colonoscopy, while the Canadian Association of Gastroenterology and British Society for Gastroenterology have included patient comfort as an additional performance indicator 6 8 9 10 .

Sedation practices vary widely between centers, regions, and endoscopists. In Ontario, Canada, over 90 % of colonoscopies are completed with sedation and among these the most common practice is to use a combination of a benzodiazepine and narcotic agent 11 . This is consistent throughout Canada. Similarly, in the United States and Australia, greater than 90 % of colonoscopies use intravenous sedation 12 . In Canada, deep sedation with propofol is estimated at 13 %, with a higher proportion in Ontario at 23 % 11 . Sedation use is highly variable in European nations. In Norwegian centers, mean sedation rate was 25 %, Portugal – 24.5 %, and UK – 94.6 % 12 . Agents used for sedation differ from country to country.

There exist a variety of well-validated patient comfort scales ranging from generic to colonoscopy specific 13 14 15 16 17 18 . The Colorado Numerical Pain Scale 16 was developed in 2002 and consists of a 0 – 5 nurse assessed pain scale, with high inter-rater reliability. It is a global rating, which is simple and easy to use, but fails to record frequency and duration of pain. Vargo et. al . developed a Patient/Clinician Satisfaction with Sedation Instrument (PSSI, CSSI), with good correlation between PSSI and CSSI scores 18 . However, it is a 16-item score, making it difficult to implement in routine clinical use. The La Cross (WI) Intra-Endoscopy Sedation Comfort Score (L-WISC) 17 , is a 4-point scale examining comfort in sedation. It showed good inter-rater reliability between physicians and nurses, but poor correlation with patient scores. It is a global rating and does not describe intensity of pain, or differentiate between frequency and duration of pain. The Nurse-Assessed Patient Comfort Score (NAPCOMS) was validated in the United Kingdom and Canada and developed in centers using minimal to moderate sedation, making it the most representative tool for our patient population 15 . During development of NAPCOMS, there was good correlation between NAPCOMS and endoscopist ratings, as well as NAPCOMS and patient ratings (both immediately post procedure and one week later), facilitating a one-time measurement. It records intensity, frequency, and duration of pain, while also measuring level of sedation and perceived global comfort.

The Hawthorne effect is a phenomenon that the very monitoring of people’s behavior affects that behavior. It was originally described in 1933, when employees at the Western Electric telephone manufacturing company were found to have higher productivity when intensely supervised. It has since been formally recognized in the field of behavioral sciences and can be found throughout research. The Hawthorne effect has primarily been investigated in quality control and endoscopy, in particular with respect to polyp and adenoma detection rates 19 20 21 22 23 .

The aim of this study was to determine whether the introduction of a patient comfort score would affect the amount of sedation administered in patients undergoing routine outpatient colonoscopy.

Patients and methods

The study consisted of 3 phases. In Phase One (P1), NAPCOMS was introduced to the endoscopy unit without informing endoscopists. In Phase Two (P2), endoscopists were informed that patient comfort was being monitored and recorded. Data from P1 and P2 were compared to see if there were any differences. The objective of Phase Three (P3) of the study was to make the Hawthorne effect more pronounced by informing endoscopists of group and individual performance data on a monthly basis.

Population

All patients undergoing outpatient colonoscopy at Hotel Dieu Hospital, in Kingston, Ontario were included in the study from a period of January 2014 to April 2014, and from January 2015 to June 2015. The Hotel Dieu Hospital Endoscopy Unit is an ambulatory endoscopy unit staffed by dedicated endoscopy nurses. The colonoscopy patient population is varied and indications include screening, surveillance, symptom assessment and therapeutics (stricture dilation, large polyp endoscopic mucosal resection). Conscious sedation is practiced at our center, with the standard of care being intravenous midazolam and fentanyl, administered by nurses under the guidance of the endoscopist.

The modified NAPCOMS was introduced to the endoscopy unit staff in November 2013, with 2 seminars prior to live use to ensure adequate nursing training in assessment and data collection. Twelve endoscopists were involved in the study (11 in P1 and P2, and 12 in P3), consisting of 8 gastroenterologists and 4 general surgeons. Apart from the addition of an additional endoscopist in P3, there were no changes in endoscopists. All endoscopists were experienced (more than 100 colonoscopies per year) and most complete greater than 200 colonoscopies per year. Eight gastroenterologists and 2 general surgeons have taken the Canadian Association of Gastroenterology Skills Enhancement in Endoscopy Train the Colonoscopy Trainer course, which aims to teach endoscopists how to effectively supervise and instruct trainees. One of the objectives of this course is addressing variations in technique for insertion of the colonoscope, with aims of both increasing success and patient comfort during the procedure.

Design

Phase One and Two of the study ran for a 16-week period from January to April 2014. Phase One was blinded to the endoscopist; they were unaware they were being monitored. This was achieved by integrating NAPCOMS into routine nursing forms. In Phase Two, endoscopists were informed verbally and via email by the Medical Director of Endoscopy that patient comfort was being assessed and data collected for an additional 8 weeks. At the time of disclosure, the score and it’s components were explained to endoscopists. In Phase Three, group means and ranges were included with individual scores to potentiate the Hawthorne effect. No additional instructions were provided otherwise.

Phase Three was conducted from January 2015 to June 2015. Sedative use, NAPCOMS results, as well as adjunctive data such as patient position changes and use of abdominal pressure was collected on a monthly basis and summarized. Physicians received, via email, monthly performance reports that included personal results as well as monthly group data (mean score, sedation used, and position changes).

Sample size

A sample size of 50 is recommended as a minimum requirement for testing associations or drawing inferences 24 , especially in a study such as this which was used to gather new baseline data. Sample size calculations to assess changes in fentanyl doses suggested 35 – 200 patients be included in each group to allow for detection of difference of 20 – 30 %, a figure we concluded would be clinically relevant. Based on the results from P1 and P2, which showed little variation between groups in sedation and NAPCOMS, a calculated sample size of greater than 10,000 cases would have been required to achieve statistical significance in NAPCOMS scores in P3. However, we elected to proceed with P3 over 6 months to assess whether direct reporting of results to endoscopists would bring about the changes we had expected between P1 and P2.

Outcomes

The primary outcome was a change in the total amount of sedation used; between P1 and P2, and in P3, over the duration of the study. Secondary outcomes were a change in NAPCOMS or adjunctive measures (position changes, abdominal pressure), over the same time periods described. A change in NAPCOMS could either be for each individual component (pain, sedation, global tolerability), or a composite pain score. Position changes were recorded as binary and as a mean, while abdominal pressure was only a binary measurement. During the study, we also looked at differences among groups, including general surgery and involvement of trainees.

The Nurse-Assessed Patient Comfort Score (NAPCOMS) was adopted to determine patient comfort during colonoscopy. NAPCOMS is a composite pain score that includes intensity, frequency, and duration, on a scale of 0 to3, with 3 being worse 15 . Sedation and global tolerability are treated separately, on a 0 to 3 scale. A NAPCOMS score of six or greater correlates with a patient rating of moderate to severe discomfort. The NAPCOMS score sheet was modified to collect additional data, including trainees, type of colonoscope used, patient demographics, complications, sedative use, procedure duration, and adjunctive measures such as position changes and abdominal pressure ( Appendix 1 ).

Appendix 1 MODIFIED NURSE-ASSESSED PATIENT COMFORT SCORE (NAPCOMS) Trainee? Yes/no Scope used: Adult/ Pediatric, other_______________

Patient
1) Age
2) Gender	Male	Female
3) Previous colonoscopy	Yes	No	If yes to (4) or (5), please list:
4) Previous bowel, bladder, or pelvic surgery	Yes	No

Statistics

Data were imported into SPSS (version 22, IBM, Armonk, NY, USA). Demographic characteristics of the patient population were described using means and standard deviations for continuous data, and frequencies and percentages for categorical data. Continuous data were also plotted to assess the normality of the underlying distributions. Most were acceptable, but sedative administration was mildly skewed so non-parametric tests were used to confirm the findings of the parametric tests. Repeated measures ANOVA was used to assess patterns over time, with the endoscopist added in as a factor to identify any between-endoscopist effects. Differences in sedative administration prior to and after introduction of NAPCOMS were assessed using the student’s t-test, and confirmed using the Mann-Whitney U.

Ethics approval was obtained and renewed from the Health Sciences Research Ethics Board from Queen’s University.

Results

One hundred and ninety-five procedures were documented in P1, and 201 in P2. Of these, 343 were completed by GI, and 53 by GS. Baseline characteristics between the groups were similar ( Table 1 ). In P3, 932 cases were documented; GI-773, and GS-159.

Phase One and Two baseline characteristics.

Parameter	Total (n = 396)	Phase One (n = 195)	Phase Two (n = 201)	P
Male	186 (47.8 %)	97 (50.5 %)	89 (45.2 %)	0.29
No trainee	288 (77.6 %)	140 (74.9 %)	148 (80.4 %)	0.20
Adult scope	324 (89.8 %)	163 (91.6 %)	161 (88.0 %)	0.49
Previous Colonoscopy	235 (62 %)	111 (60.7 %)	124 (63.3 %)	0.60
Previous Surgery	124 (36.3 %)	53 (32.7 %)	71 (39.4 %)	0.20

Phase One and Two were designed to test whether the introduction of a patient comfort scoring system would affect endoscopist behavior on sedative administration. Midazolam, fentanyl and hyoscine butylbromide were the only sedative agents used and there were no statistically significant differences in the amount of sedative use between P1 and P2, our primary endpoint. There were no statistically significant differences in NAPCOMS. Average overall NAPCOMS score was 3.07. Global tolerability ( P  = 0.052) showed increased comfort in P2, but did not reach statistical significance. There was a higher proportion of cases where position changes were used in P2 (blinded n = 103, open n = 163, P  = 0.003), however, the mean number of position changes did not reach statistical significance by group (blinded 3.01 ± 1.84, open 3.41 ± 2.25, P  = 0.19). There was a statistically significant increase in the use of hyoscine butylbromide in P2, however this is accounted for entirely by 1 physician.

Phase Three collected and reported results to endoscopists on a monthly basis. Data was analyzed across months, by endoscopist, and comparing the first month to all other months in the study. Endoscopist data showed many differences, reflecting different physician preferences and are not reported here. There were no discernable patterns in the data. Comparison of monthly data showed a decrease in fentanyl use as the study progressed ( P  = 0.035, 75.57 mcg, 95 % CI 73.50, 77.64), and variation in midazolam use ( P  < 0.001), number of position changes ( P  = 0.001), and procedure duration ( P  = 0.004) but no trend towards increased or decreased amount ( Fig. 1 ). This correlated with heterogeneous results between months in NAPCOMS global tolerability score. Composite NAPCOMS, sedation score, and use of abdominal pressure showed no statistically significant differences. Average NAPCOMS score was 2.97.

Fig. 1

 Phase Three procedural documentation – sedative use, number of position changes, and procedure duration.

In all phases, we compared results between gastroenterologists and general surgeons. Combining all data in P1 and P2, general surgeons used higher amounts of midazolam (gastroenterologists 1.45 ± 0.56 mg, general surgeons 2.08 ± 0.67 mg; P  = 0.001), although there were no differences in NAPCOMS. Gastroenterologists had a higher proportion of position changes ( P  < 0.001) but the mean number showed no differences. In P3, we once again saw increased amounts of midazolam use (gastroenterologists 1.51±0.85 mg, general surgeons 1.69 ± 0.57 mg; P  = 0.001) and fewer position changes both in proportion ( P  = 0.002) and mean ( P  = 0.01) used by general surgeons. Gastroenterologists used higher amounts of fentanyl (gastroenterologists 76.52 ± 30.92 mcg, general surgeons 70.50 ± 32.59 mcg; P  = 0.037), a result which was not observed in P1/P2 (gastroenterologists 87.21 ± 37.38 mcg, general surgeons 88.73 ± 40.41 mcg; P  = 0.79). There remained no statistically significant differences in NAPCOMS. There was no difference in procedure duration in any phase ( Fig. 2 ).

Fig. 2

 Gastroenterology compared to general surgery procedural documentation – sedative use, number of position changes, and procedure duration.

Using combined P1 and P2 data, we compared colonoscopies performed by trainees versus no trainee. Baseline characteristics were similar between groups. Trainees used significantly more midazolam (no trainee 1.52 ± 0.57 mg, trainee 1.69 ± 0.72 mg; P  = 0.02) and fentanyl (no trainee 85.47 ± 29.20 mg, trainee 96.61 ± 56.50 mg; P  = 0.01), while their procedures were longer (no trainee 30.75 ± 12.59 minutes, trainee 43.92 ± 15.51 minutes; P  < 0.001), and NAPCOMS scores worse in all domains except level of sedation ( Fig. 3 ).

Fig. 3

 Comparison of no trainee and trainee procedures. a NAPCOMS.  b Sedative use, position changes, and procedure duration.

Discussion

The aim of this study was to determine if the introduction of a patient comfort scoring system would affect physician behavior in regard to sedative use. Phase One and Two assessed endoscopist-blinded and – aware behavioral response, and Phase Three the effect of monthly feedback. Conventionally, quality control measures in colonoscopy have included CIR, ADR, and adherence to guidelines. Patient comfort plays an important role in patient satisfaction and is also a measure of endoscopic quality and technique 25 . Healthcare delivery is increasingly patient focused and patient comfort scores such as NAPCOMS help us quantify the patient experience.

In the first 2 phases of our study, there was no difference in sedative administration or NAPCOMS scores between pre- and post-intervention periods. A potential explanation was that monitoring was unobtrusive and not recognized by the endoscopist. Endoscopists were only informed one time that they were being monitored in the endoscopist aware period, and thus may have quickly forgotten. To test this theory, P3 fed back each endoscopists individual data on a monthly basis plus group means and ranges. There was a statistically significant decrease in fentanyl use and duration of pain. While statistically significant, the clinical significance was minimal, as evidenced by no overall change in NAPCOMS. Direct comparison among individual endoscopists was not provided as feedback. Doing so may have produced a more significant Hawthorne effect.

We expected to see an increase in medication use and correspondingly, a decrease in NAPCOMS. This did not occur in any comparison. We would have required greater than 10,000 exams to look for miniscule differences in NAPCOMS. NAPCOMS is a composite of intensity, frequency, and duration of pain for a total score of nine. Level of sedation and global tolerability were assessed separately in the original paper. In P1 and P2, average overall NAPCOMS was 3.07 and in P3, 2.97. This is lower than the average reported in the original paper (3.2) 15 . Results from the use of NAPCOMS have not been widely published, and therefore, we are unaware of a normal range. If our scores are low compared to this range, this would affect our results.

Comfort scores have been correlated with performance. In 2010, comfort scores using nurse-reported comfort levels were compared to CIR and polyp detection rate 26 . Endoscopists with higher patient comfort scores were found to have higher polyp removal and CIR. Improved comfort scores were associated with less sedative use. In contrast, work presented in abstract form showed “standard” doses of midazolam resulted in higher CIR and polyp detection rate compared to doses less than 2 mg 27 28 . Notably, standard doses in the abstract were 2 mg, which is higher than the average doses seen at our center.

Physician specialty

Practice differences between general surgery and gastroenterology were compared in our study. We consistently found general surgeons used fewer position changes and in P3, less fentanyl and more midazolam. While statistically significant, clinically, there was no effect on NAPCOMS.

There remains debate in the literature among the quality of endoscopy among different stakeholders. A study comparing practice habits between nurses, GS, and GI fellows found no differences in effectiveness or patient satisfaction in flexible sigmoidoscopy 29 . A study by Mehran et. al . comparing general surgeons, colorectal surgeons, and gastroenterologists showed no difference in complication rates, and shorter procedure time among GS 30 . On the contrary, a cross sectional study of endoscopists from seven Montreal (gastroenterologists 36, general surgeons 6, n = 2113) and 2 Calgary (gastroenterologists 31, general surgeons 5, n = 538) endoscopy centers found in the Montreal center, general surgeons were half as likely to remove polyps compared to gastroenterologists 31 . Adjusted odds ratio for general surgeons compared to gastroenterologists was 0.48 (95 % CI: 0.32 – 0.71) and 0.73 (95 % CI: 0.43 – 1.21) for Montreal and Calgary, respectively. Our study showed differences in practice habits between gastroenterology and general surgery, but no difference in overall quality.

Position changes

Position changes are used in colonoscopy to improve luminal distention, allowing for better visualization. Several studies have demonstrated the benefit of position changes on ADR, which reflects quality of colonoscopy. East et. al . showed that position changes improved luminal distention during withdrawal but did not assess ADRs 32 . Three randomized trials compared colonoscopy performed in left lateral decubitus to dynamic position changes on ADR and found a 9.8 % 33 , 18 % 34 , and 9.4 % 35 absolute increase in ADR in the group with position changes. Position changes can also be used to facilitate insertion of the colonoscope, resulting in better tolerance. There was no difference in position changes in any phase. Position changes were used more frequently by GI in our study but resulted in no clinical differences.

Trainees and endoscopy

Most major gastroenterology societies have released guidelines on principles of endoscopy training and minimal competency procedural thresholds. Despite this, there have been few guidelines specific to how endoscopists should be trained. Similarly, there have been limited studies examining patient comfort in endoscopy when comparing trainee procedures to attending staff alone. Our study showed that trainees used more sedation, had longer procedures, and resulted in more patient discomfort compared to staff alone. Our group has previously reported extension of procedure time by 50 % in ambulatory colonoscopy when trainees are involved 36 .

Data comparing patient comfort in trainees to staff procedures are conflicting and arise mainly from validation of training techniques and quality assurance studies. Flexible sigmoidoscopy completed by senior internal medicine residents was compared to staff, and while procedures involving residents were longer, satisfaction and comfort did not differ between groups 37 . Additional studies in colonoscopy have shown that trainee involvement have no effect on satisfaction, pain, or comfort 38 39 . In 1 study, trainee procedures were longer, used more sedation, and had lower CIR rates 39 . In contrast, a comparison of patient satisfaction among gastroenterologists, internists, and nurse practitioners found procedures involving trainees had less overall satisfaction, and more pain and discomfort 40 . A comparison of residents randomized to simulation or patient based training found both groups had higher discomfort scores compared to staff 41 . All of these studies used patient based questionnaires to determine comfort, while ours was nurse assessed. Ultimately, data from trainees to attending staff procedures remain conflicting, with our study suggesting trainee involvement may lead to more sedative use and patient discomfort. The implications of this for teaching hospitals and training programs will need to be further considered.

Trainee effects on other measures of quality have been studied to some degree. Work presented in abstract form showed that trainees had longer withdrawal time compared to staff alone 42 . This is consistent with our findings and is intuitive, given lower technical proficiency and the addition of teaching during trainee endoscopy. Several groups have looked at ADR and again, results are conflicting. Shah et. al . and Eckardt et. al . did not find any difference in ADR between staff and trainees 43 44 . In Eckardt, there was a trend towards increased ADR in the staff group but this was not significant (19.3 % vs 14.9 %, P  = 0.27). Other studies have shown improved adenoma detection with trainee involvement 45 46 47 , with the prevailing theory that two sets of eyes are better than one. Notably, in the study by Peters et. al. , level of training was associated with a higher polyp detection rate and ADR, with third-year fellows having significantly better performance compared to their junior colleagues. Although withdrawal time increases ADR, these results show that experience also plays a large factor.

It is widely accepted that trainees increase procedure time in endoscopy. Studies to date are conflicting in regard to trainee effects on endoscopy quality and patient comfort. While our study showed trainees used more sedation and had worse comfort scores, we cannot directly relate this to quality. Additional studies would be required to correlate our findings with established colonoscopy quality indices.

Limitations

Our study focused mainly on sedative use and perceived patient comfort during procedures. We did not measure established markers of quality in endoscopy and doing so would help correlate our findings with previous studies. As this was a single center study, there was a relatively low volume of colonoscopies included. This was particularly amplified in the general surgery analysis, which had a minimal case volume compared to gastroenterology. The small general surgery case volume may affect the comparisons presented in this paper.

Conclusion

Introduction of a patient comfort scoring system resulted in a statistically significant decrease in fentanyl use, although with minimal clinical significance, as there was no change in NAPCOMS. Additional studies are required to determine the role of patient comfort scores in quality control in endoscopy. Procedures completed with trainees used more sedation, were longer, and had worse NAPCOMS scores, the implications of which, for teaching hospitals and training programs, will need to be further considered.

Appendix 1

MODIFIED NURSE-ASSESSED PATIENT COMFORT SCORE (NAPCOMS)

Trainee? Yes/no Scope used: Adult/ Pediatric, other _______________________

NAPCOMS

Domain	Item	0	1	2	3
Pain	1) Intensity	None or minimal	Mild	Moderate	Severe
	2) Frequency	None	Few (1 – 2 episodes)	Several times (3 – 4 episodes)	Frequent (> 4 episodes)
	3) Duration	None	Short duration (episode < 30 seconds)	Moderate duration (30 seconds – 1 minute)	Long duration (episodes last ≥ 1 minute)
Sedation	Level of consciousness	Alert	Sleepy but initiates conversation.	Responds only when asked or stimulated.	Unresponsive or only responds with pronounced stimulation.
Global	Tolerability	Very well tolerated.	Reasonably well tolerated.	Just tolerated.	Poorly tolerated.
Complications	VasovagalEvent	HTN	Tachycardia	Other (please list):

PROCEDURAL DOCUMENTATION

Sedative	Midazolam (mg)	Fentanyl (mcg)		Buscopan (mg)		Other: ______________
Duration (mins):	Number of Position Changes:	Abdominal Pressure Applied			CO 2 Insufflation Used
		YES	NO		YES		NO