Lack of recall after sedation for cataract surgery and its effect on the validity of measuring patient satisfaction.

BACKGROUND: We evaluated the validity of assessing patient satisfaction with the sedation regimen among patients being discharged 45 min after receiving midazolam. If most patients do not have recall, then the sedation cannot be considered complete at the time of evaluation.
METHODS: In this prospective cohort study, 20 patients underwent cataract surgery with nurse-administered midazolam and fentanyl. The 11-item Iowa Satisfaction with Anesthesia Scale was administered  30 min after sedation in the recovery room. Recalled items were evaluated the next morning.
RESULTS: Eleven patients recalled 0 themes, 4 recalled 1, 4 recalled 2, and 1 recalled 3 themes. Thus, 15/20 patients (75%) recalled 0 or 1 of the 11 themes (P = 0.021 versus half the patients). The 95% one-sided lower confidence limit for 0, 1, or 2 themes was 80% of patients (P < 0.001 versus half). Patients who received less midazolam recalled more themes (Kendall's τb = 0.43, P = 0.039).
CONCLUSIONS: Evaluating patient satisfaction with sedation shortly after admission to the post-anesthesia care unit is invalid because of a lack of recall; the sedation/amnesia is ongoing. Patient comfort may be assessed, but comfort is not synonymous with satisfaction; 'satisfaction' implies presence of recall. Because we studied sedation with low doses of midazolam and fentanyl, the same conclusion reliably would apply to larger doses of anxiolytics administered intraoperatively. The results match previous findings that when patients receive preoperative midazolam prior to meeting the anesthesiologist, even if the patient fully answers questions, they may have negligible recall of having met the anesthesiologist.

Introduction

Nearly all patients undergoing cataract surgery at Mayo Clinic Jacksonville receive low-dose intravenous sedation with midazolam (and fentanyl analgesia) while being monitored intraoperatively by registered nurses (i.e., patients receive nurse-administered sedation). Patient satisfaction with the care they receive can be measured using the reliable and valid Iowa Satisfaction with Anesthesia Scale [1-8].

We originally, and unsuccessfully, sought to perform a prospective cohort study in cataract surgery patients to examine the association between the doses of midazolam (and fentanyl) that patients receive, and scores on the Iowa Satisfaction with Anesthesia Scale. All patients included in the study had their satisfaction with sedation assessed soon before (e.g., within 10 min of) leaving the outpatient surgery department, consistent with the development and use of the Iowa Satisfaction with Anesthesia Scale [1]. However, during the pilot study of 10 patients (which was performed to choose the appropriate sample size to measure satisfaction), we observed that patients were being discharged from the facility within 45 min after receiving midazolam and fentanyl. That time course was similar to the periods studied by Chen et al. [9], wherein when patients received midazolam in the preoperative holding area, most had complete amnesia of the operating room events. The findings [9] were important because the implication would be the lack of validity in assessing patient satisfaction with individual anesthesiologists who provide exclusively operating room care. We hypothesized that it may be invalid to assess patient satisfaction with sedation among patients so soon after receiving midazolam, because if most patients do not have recall, then the sedation cannot be considered complete at the time of the evaluation. We performed a prospective cohort study to test the hypothesis that most cataract surgery patients (i.e., significantly greater than half of patients) have recall of only negligible portions of the Iowa Satisfaction with Anesthesia Scale items that they answered before discharge from the facility.

Materials and Methods

The Mayo Clinic Investigational Review Board approved this study (application 17 010108), and then patient enrollment began. All cataract surgery cases were performed in a single operating room of the Mayo Clinic Jacksonville outpatient surgery department. The phase I and phase II post-anesthesia care units function interchangeably; henceforth, they are referred to as the ‘recovery room.’ The patients excluded were the few (≅11% [4/37]) that were selected by the surgeon to receive deep sedation or general anesthesia.

During a preoperative visit, standard discharge instructions were reviewed with each patient, including wearing the eye patch that was to be placed in the operating room until the first postoperative follow-up visit with the ophthalmologist. We took advantage of this uniform approach to have each patient serve as his/her own control.

In the preoperative area, patient informed consent was obtained from each patient by a research staff member. Patients were instructed that they would receive a survey to assess their level of satisfaction with the sedation they will receive intraoperatively based on symptoms they experienced during the procedure. (i.e., the Iowa Satisfaction with Anesthesia Scale). The patients were told that they would be asked to complete the survey shortly after entry to the recovery room. Patients also were advised that they would receive a phone call the next day from one of the members of the anesthesia department with follow-up questions.

In the recovery room, the patients were informed that they would be provided with 11 statements, and they would need to choose to either agree or disagree with the options provided. For each statement with which they agreed, they would be asked whether they agreed ‘slightly,’ ‘moderately,’ or ‘very much.’ As patients had undergone ophthalmologic surgery, they were also given the option to have the questions read to them rather than reading the questions themselves. The patients were advised that within 24 h after discharge, they would receive a telephone call asking them: ‘Please recite as many of the 11 questions you were asked yesterday.’

The Iowa Satisfaction with Anesthesia Scale was used for several reasons. The scale assesses satisfaction with the anesthetic/sedation itself, including satisfaction with sedation; based on qualitative methods and quantitative correlations, surgical patients consider this separate from satisfaction with the preoperative and postoperative periods [1,4]. The scale has good test-retest reliability and internal consistency [1-4]. Validity of the scale has been demonstrated based on correlations with other variables such as anesthesiologists’ predictions of patients’ responses [1]. We are not aware of the existence of another scale developed for measuring patients’ satisfaction with sedation for surgery including cataract procedures [4]. The full scale is listed, and its use described, at the hyperlinks in References [10] and [11], respectively.

During the follow-up phone call, the patients were asked to recall as many of the 11 questions they were asked the previous day. They were also asked to recall the discharge instructions about the eye patch that they were given both during the preoperative visit and before discharge from the recovery room.

All responses and times were recorded for both the survey on the day of surgery, and the follow up questions on the day after surgery. Respondents’ answers to questions were transcribed exactly as spoken by the patients. Additional data collected preoperatively included patient sex, age, dosages and times of intravenous sedation medications (midazolam and fentanyl), surgery start and end time, and recovery start and end time.

Sample size selection and statistical analyses

The sample size was chosen based on a one-sided test using data from the 10-patient pilot study. We treated the 11 items in the Iowa Satisfaction with Anesthesia Scale [1-3] as being themes, such that any response potentially related to an item was considered related (e.g., patient referring to being asked about pain levels was treated as referring to the item ‘I felt pain,’ even though the patient may be recalling a recovery room nurse asking about their pain). Thus, the design was biased, purposefully, to overestimate explicit recall of the Iowa Satisfaction with Anesthesia Scale’s 11 items (i.e., the design was biased, purposefully, to reject our studied hypothesis). The identification of recall of themes was made independently by all three authors using the transcribed responses. There was unanimous agreement for all patients and potential themes. Among the 10 pilot patients, there was 1 patient who may have had recall of 2 of 11 themes, 1 patient who had recall of 1 of 11 themes, and 8 patients had no recall of any themes. We therefore planned to compare the incidence of recall of > 2/11 themes to 50% using the binomial test [12] (The 50% tests our hypothesis that ‘most’ patients recall only negligible portions of the Iowa Satisfaction with Anesthesia Scale). Given that we observed 10/10 patients (100%) with recall of 0, 1, or 2 themes, we conservatively designed the prospective study to test whether approximately 80% (i.e., 8/10 patients) would have recall of 0, 1, or 2 themes versus > 2/11 themes. With a subsequent sample size of n = 20 patients (i.e., no overlap of patients with our pilot data), there would be approximately 80.4% statistical power for the one-sided test of 80% as compared with 50%. We planned to repeat analyses using 0 or 1 recalled themes versus 2/11 themes. These and all other P-values were calculated using exact methods (StatXact-11.1, Cytel, Inc., USA).

Pairwise comparison was made between the recall of 0, 1, or 2 themes versus > 2/11 themes and recall versus no recall of the instruction to wear eye patch until the first postoperative follow-up visit with the ophthalmologist. We used Barnard’s unconditional test for equality of two related proportions, with a two-sided test for difference in recall. Calculations also were performed comparing 0 or 1 theme versus ≥ 2/11 themes.

Associations between the number of themes recalled and demographic variables were tested using Kendall’s τb correlation. The P values were two-sided.

Results

Among the 20 patients studied, 11 recalled 0 themes, 4 recalled 1 theme, 4 recalled 2 themes, and 1 recalled 3 themes. Thus, among the 20 patients, 15 (75%) recalled 0 or 1 of the 11 themes (P = 0.021 versus half the patients). The 95% one-sided lower confidence limit for recall of 0 or 1 themes was 55% of patients. Among the 20 patients, 19 (95%) recalled 0, 1, or 2 of the 11 items. The 95% one-sided lower confidence limit for 0, 1, or 2 themes was 80% of patients (P < 0.001 versus half). The 19/20 was comparable to the 10/10 patients observed in our pilot (Fisher’s exact test P = 0.99).

Among the 4 patients who recalled 1 theme (n = 4), 4 patients who recalled 2 themes (n = 8), and 1 patient who recalled 3 themes (n = 3), there was a total of 15 recalled themes. Among the 15 themes, 6 were of pain, 5 of hot or cold feeling, 3 of nausea or vomiting, and 1 of feeling relaxed. Using Kendall’s τb correlation, responses to these themes were unrelated statistically to the numbers of overall recalled themes (Table 1).

Table 1.

Associations between Demographic and Clinical Variables and the Number of the 11 Iowa Satisfaction with Anesthesia Scale[*] Items Recalled among the 20 Patients

Variable	Median [IQR]	Kendall’s τb[†]	Two-sided P value[‡]
Male	0 [0, 1]	–0.19	0.34
Age	72 [68, 76]	0.16	0.99
Duration of surgery	16 [13, 27]	–0.13	0.49
Minutes in recovery room	18 [14, 23]	–0.14	0.46
Days from July 1, 2018	44 [32, 86]	–0.30	0.13
Duration of phone call (min)	3 [3, 4]	0.27	0.19
Fentanyl (μg)	25 [25, 50]	–0.50	0.024
Midazolam (mg)	1.0 [0.75, 1.0]	–0.43	0.039
ISAS score	2.91 [2.05, 2.91]	0.12	0.56
‘I felt pain’	3 [–1, 3]	0.07	0.75
‘I felt pain during surgery’	3 [–1, 3]	0.10	0.66
‘I was too cold or too hot’	3 [3, 3]	–0.39	0.10
Fentanyl to ISAS started (min)	32 [28, 34]	0.17	0.38
Midazolam to ISAS started (min)	32 [29, 34]	0.11	0.56
Midazolam to phone call (h)	24.4 [21.8, 26.0]	–0.08	0.67

ISAS: Iowa Satisfaction with Anesthesia Scale, with the overall score and its components each ranging from –3 (agree strongly) to +3 (disagree strongly).

The Kendall’s τb were calculated between the numbers in the first column and the number of themes among the 11 Iowa Satisfaction with Anesthesia Scale items potentially recalled. The n = 20 patients for all rows. Fig. 1, the abstract, and the Results include the findings from the 8th row. Specifically, ‘patients who received less midazolam recalled more themes (Kendall’s τb = 0.43, P = 0.039).’ All P values are two-sided and exact (StatXact-11). The latter is important because there were few different categories of numbers of themes recalled: 0, 1, 2, or 3 (see Results). In addition, in the first row, there are only two values of the potential predictor being tested.

The P values are unadjusted for the multiple comparisons, and therefore should be judged as having a substantive chance for a Type I error.

Patients were used as their own control. The patients were asked about their recall of the instruction to wear the eye patch until their first postoperative follow-up ophthalmology appointment; as above, this instruction was provided not only upon discharge, but also preoperatively. All 5 patients who recalled 2 or 3 Iowa Satisfaction with Anesthesia Scale themes also correctly recalled the eye patch instructions. There were 5 patients who both recalled 0 or 1 theme and incorrectly recalled the eye patch instructions. The other 10 patients recalled 0 or 1 theme but remembered the eye patch instructions correctly. Thus, the total of 15 of 20 patients who recalled the eye patch instructions was significantly greater pairwise than the 5 of 20 patients who recalled 2 or 3 themes (P < 0.001). Consequently, too, the 15 of 20 patients who recalled the eye patch instructions was greater pairwise than the 1 of 20 patients who recalled > 2 themes (P < 0.001).

Table 1 includes 15 demographic variables. Patients who received less midazolam recalled more themes (Fig. 1; Kendall’s τb = 0.43, P = 0.039).

Fig. 1.

Observed relationship among the 20 patients between the dose of midazolam and the number of the 11 themes recalled in the Iowa Satisfaction with Anesthesia Scale. The midazolam was administered a median of 32 min (25th percentile 29 min, 75th percentile 34 min) before the questions were asked. The patients were asked to recall the items in the scale during the phone call made the next morning. We treated any response related to the theme of the item to be recall.

Discussion

The observation that elderly patients receiving both a benzodiazepine and opioid had impaired recall is unsurprising based on previous volunteer results [13]. However, the importance and novelty of our current findings are the timing and the corresponding implications for the validity of administration of satisfaction scales and patient instructions. Despite the practical relevance of our finding for application to clinical trials, the scenario envisioned by our study was not recognized during the systematic review and consortium meeting on intraoperative sedation including patient-centered endpoints for the US Food and Drug Administration [4,5].

Implications

Our results show that assessing patient satisfaction with intraoperative sedation within a few minutes after entry into the recovery room is invalid. Comfort may have been assessed, but comfort is not synonymous with satisfaction, as ‘satisfaction’ implies the presence of recall. If most patients do not have recall, the sedation cannot be considered complete at the time of such assessment. Because we studied nurse-administered sedation with low doses of midazolam (Fig. 1) and fentanyl, the same conclusion reliably would apply to larger doses of anxiolytics administered by anesthesiologists or nurse anesthetists while providing monitored anesthesia care. The findings are directly applicable to clinical trials evaluating patient-centered outcomes that involve procedural sedation for surgical procedures [3-5].

Postoperative phone calls have been reported to have low response rates in some populations (e.g., France) [14]. Although texting can be used effectively to obtain basic postoperative information from surveyed patients (e.g., numerical pain ratings) [14], scales reliably measuring satisfaction with sedation cannot be administered in this fashion [4,5]. Organizations may attempt, as the current investigators at Mayo Jacksonville originally did (see Methods), to obtain low non-response bias (i.e., improve patient response rates) by administering the satisfaction survey just before patients were discharged from the ambulatory center. That was precisely how the Iowa Satisfaction with Anesthesia Scale was developed [1,15,16]. Our results show the importance of the timing of administration of the scale. When used previously, there usually was an intervening period ≅ 3 h between the last dose of sedatives administered to patients and the time when the patient completed the scale. In contrast, in the current study, the administration of the Iowa Satisfaction with Anesthesia scale was done within a few minutes of the patients’ arrival to the recovery room, because the patients were discharged from the facility soon afterwards.

Previously, in the United States, there has been consideration of including patient satisfaction when evaluating the medical care provided by individual anesthesiologists [9,17]. The current results are important because they document that even when patients are fully answering questions postoperatively, they may have negligible actual recall of the care they received. An experience without recall may be enjoyable and may lack discomfort (or, on the other hand, may be negative). However, satisfaction requires recall by the patient.

Limitations

Our results relate to patient satisfaction as a measure of the quality of care, but not as influencing the number of patients presenting to a healthcare facility (i.e., marketing) [18]. The importance of satisfaction is that of the surgeons, because surgeons return to the ambulatory surgery center much more frequently compared with patients [18]. For the specific procedure studied of cataract surgery, patient satisfaction does matter, but indirectly by influencing surgeon experience. The relevant dimensions of patient satisfaction are not related to the anesthetic care itself, but to case scheduling: the ability to provide the surgical procedure on a day chosen by the patient; to receive care at a single site for both examination and surgery; to combine the examination and surgery into a single visit instead of multiple visits; and to have surgery in the morning instead of the afternoon [19].

The sample sizes required to compare patient satisfaction between groups typically would be ≅ 70 patients in each of 2 groups [3]. Our studied sample size was chosen deliberately based on our primary endpoint of testing recall, 0, 1, or 2 themes versus > 2/11 themes. We did not design our study to measure patient satisfaction per se, and we could not deliberately do so because of the chosen sample size. There have been previous studies that have assessed patient satisfaction with cataract surgery using the Iowa Satisfaction with Anesthesia Scale [2,6,7].

In our current investigation, we explored the relationship between the dose of midazolam and recall of the items in the Iowa Satisfaction with Anesthesia Scale. Fig. 1 shows that increasing doses of midazolam were associated with recall of fewer themes from this scale. That finding complements results from Chen et al. [9], who found that increasing doses of midazolam administered in the preoperative holding area were associated with greater incidences of complete amnesia of operating room events; see their Table 2. Likewise, Miller et al. [13] examined the timing of amnesia after 0.07 mg/kg and 0.10 mg/kg doses of midazolam. Among pictures shown to patients at ≅ 30 min, matching our data (Table 1), approximately 45% and 75% of patients, respectively, lacked recall the following morning. The similarity of our findings to those reported by Chen et al. [9] and Miller et al. [13] increases confidence in the validity of our findings. However, in the current study, we cannot separate the amnestic effects of midazolam from those of midazolam plus fentanyl, because all patients received both drugs (a minimum 0.5 mg midazolam and 25 μg fentanyl).

We purposefully biased our estimates of recall by including any item possibly related to a theme in the Iowa Satisfaction with Anesthesia Scale. For example, if a nurse asked the patient if they had pain in their eye, and when asked about the Iowa Satisfaction with Anesthesia Scale, the patient referred to pain, then that response was treated as recall of a theme of pain. The same applied to any patient who may have been asked by a nurse if he/she was hot or cold. The fact that despite the bias, the recall was so limited increases validity of our conclusions, despite the reduction in accuracy of our numerical results.

Patient age likely was not a covariate, because we studied homogeneously elderly patients (median 72 years; Table 1). However, we therefore do not know if our current results may differ from those of younger patients. In a study of word recall, older patients (mean 67 years) recalled fewer words than did younger patients (mean 26 years) [20]. Diazepam had an equivalent effect on recall among both young and elderly patients [20].

We studied patients undergoing cataract surgery, partly because it is the most common surgical procedure nationwide [21]. In addition, we selected cases performed only during regular work hours (e.g., not at night; Table 1), matching the timing when nearly all ambulatory surgery is performed nationwide [22,23].

Conclusions

Elderly patients received low-dose midazolam, and approximately half an hour later were asked the series of questions that are part of the Iowa Satisfaction with Anesthesia Scale. The patients were sufficiently alert to be discharged a few minutes later. However, the next day, patients had negligible recollection of the items in the scale. One implication from our results is that patient satisfaction cannot validly be assessed so soon after amnestic drugs (e.g., midazolam) have been administered, even if the patient seems alert and is deemed ready for discharge; additionally, instructions about postoperative care should not be provided to the patient during that period. A second implication is that patient satisfaction with individual practitioners should not be evaluated for clinicians interacting with the patient during this period, because the patient responses will often not be based on actual recall of factual interaction with those practitioners or their statements.