Patient Perspectives on Defining Textbook Outcomes Following Major Abdominal Surgery.

BACKGROUND: The composite metric textbook outcome (TO) has recently gained interest as a novel quality measure. However, the criteria for defining a TO have not been rigorously defined and patient perspectives on the characteristics of TO are unknown.
METHODS: Patients who underwent major abdominal surgery at a single tertiary care center were administered a customized survey designed to ascertain their perspectives on defining TOs. The relationship between patient-reported and clinically defined TO rates was compared.
RESULTS: Among 79 patients who underwent gastrointestinal (51%), pancreatic (29%), hepatic (18%), or other major abdominal (3%) operations, 57% were female and 86% had an ASA class ≥3. Most patients underwent surgery for malignancy (87%) with 60% undergoing an open operation. Patients most commonly valued no mortality following surgery (96%), no reoperation (75%), and having a margin negative resection (73%) as "extremely important." In contrast, those outcomes that were most commonly valued as "not important at all" or "minimally important" were receiving a blood transfusion (24%) and not having any complications (13%). Using previously published criteria for TOs, 47 (60%) patients were classified as having a clinically defined TO; in contrast, 68 patients (86%) self-reported their outcome was textbook. Self-reported responses were concordant with clinically defined TO criteria 63% of the time (McNemar's test: S=15.2, p<0.01, evidence of disagreement).
CONCLUSION: There was significant discordance between patient-reported versus clinically defined measures of TOs, suggesting patients value other considerations beyond traditional factors when evaluating the success of their surgery. Future studies should delineate these relationships and incorporate these factors to refine TO definitions.

Introduction

Providing high-quality surgical care is of critical importance to patients, providers, hospitals, and payers. Almost 2 decades ago, the Institute of Medicine (IOM) published Crossing the Quality Chasm that sparked efforts in identifying gaps in health care quality, calling for improvements in performance and providing a rationale and framework for the redesign of the health care system.1, 2 In line with the IOM’s report, the surgical community has sought a multi-faceted approach to improve the quality of patient care, most notably by measuring outcomes and focusing efforts on improving those outcomes. One of the most successful examples of this is the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP), which is a nationally validated, risk-adjusted, outcomes-based program that measures and improves the quality of surgical care, and in turn provides participating hospitals with tools and reports to make informed decisions about improving quality of care.3

However, efforts such as ACS-NSQIP use single outcome measures, which are imperfect in that they emphasize isolated deficiencies and may not reflect multilevel processes that impact patient outcomes especially for complex procedures [4, 5]. In contrast, composite measures may be superior to individual metrics for evaluating the quality of complex surgical procedures as they more accurately reflect the majority of these pertinent parameters.6–9 In this context, recent research has focused on the novel composite metric known as a “textbook outcome” (TO) to describe when all desired perioperative outcomes have been met.4, 5, 10–19 Since a TO includes the most important outcome parameters of the surgical process, it provides an objective overview of the overall quality of care.

A weakness of current TO definitions is that they are derived from physician consensus without formal patient engagement. Integrating patients’ perspective as stakeholders to define relevant outcome metrics is essential for ensuring improving patient-centered quality of surgical care.20, 21 As little prior research has been conducted on defining patient perspective of surgical textbook outcomes, the purpose of this study was to (1) ascertain patients’ perspectives on criteria for defining a TO following major abdominal surgery and (2) assess concordance between self-reported and clinically defined rates, using previously published definitions, of TOs.

Materials and Methods

An observational cohort study was designed aiming to assess the perspectives of postoperative patients seen in the surgical oncology outpatient clinics at a single tertiary care center. Consecutive eligible patients with gastrointestinal, hepatopancreatobiliary, colorectal, or retroperitoneal pathology were offered enrollment within 3 months of their surgery date at a regularly scheduled postoperative visit. Patients were counseled that the objective of the study was to understand their views of a “textbook outcome” (explained as a composite measure that intends to assess the quality of care he or she receives) following their surgical procedure. The study was approved by the Institutional Review Board.

A customized survey was created using TO criteria from previously published studies.4, 5, 10–19 Subjective factors such as pain control and cosmetic outcome were omitted from the finalized survey given their absence in past TO definitions. The survey consisted of three major sections. The first section asked patients to rank the importance of 11 endpoints on a 5-point Likert scale ranging from “not important at all” to “extremely important” to experiencing a TO. The second section asked patients to rank 10 items in order of importance in defining a TO from 1 (least important) to 10 (most important). The last section asked patients if they believed they had a TO following surgery and if there was anything additional they believed should be considered in the definition of a TO (Figure 1). The electronic medical record was retrospectively reviewed to measure relevant clinical, pathologic, surgical, and postoperative factors.

Fig. 1

Patient survey

First, descriptive statistics were calculated using frequencies and percentages for categorical data. Medians and interquartile ranges (IQR) along with means and standard deviations (SD) were used for continuous data. Categorical variables were assessed using the chi-square test for proportions or Fishers exact test as appropriate. Second, McNemar’s test was used to compare the relationship between patient-reported and clinically defined TO rates. These clinically defined TO were based upon previously published criteria for comparable surgical procedures.11, 12, 16–18 Specifically, a clinically defined TO was defined as the absence of all of the following: positive resection margin, Clavien-Dindo22, 23 grade ≥2 postoperative complication, perioperative blood transfusion, need for unplanned invasive procedures, need for reoperation within 90 days, prolonged hospital stay >10 days, non-home discharge, hospital readmission within 90 days, and any cause mortality within 90 days.

Statistical significance was established as 2-sided p-values < 0.05. Statistical analyses were performed using SAS Version 9.4 (SAS Institute, Cary, NC). The Ohio State University Institutional Review Board approved the finalized survey and associated protocol. The study was conducted from August 2019 to April 2020 (terminated early because of limits placed on in-person clinical research due to the COVID-19 pandemic).

Results

Overall, there were 84 patients in total who were asked to take the survey with 79 saying yes. Among 79 enrolled patients, the median age was 60 years (IQR: 51–66) and a majority were female (57%). Patients were more commonly of white race (92%) and either overweight (29%) or obese (42%). Comorbid conditions were present in a majority of patients, with 86% having an American Society of Anesthesia (ASA) class three or greater. A majority of patients underwent gastrointestinal surgery (51%), followed by pancreatic (29%), hepatic (18%), or other major abdominal (3%) operations. Most patients underwent surgery for cancer (87%) with 60% undergoing an open operation. Complete patient characteristics are reported in Table 1.

Table 1

Demographic, clinical, operative, and postoperative characteristics undergoing major abdominal surgery

Variable	Overall
Demographics
Age, years, n (%)
<65	51 (65)
≥65	28 (35)
Gender, n (%)
Male	34 (43)
Female	45 (57)
Race, n (%)
White	73 (92)
Non-White	6 (8)
Patient and operative characteristics
BMI, n (%)
Normal	23 (29)
Obese	23 (29)
Overweight	33 (42)
ASA class
2	11 (14)
3	66 (84)
4	2 (3)
Albumin, g/dl, n (%)
<3.5	21 (27)
≥3.5	58 (73)
Cancer diagnosis, n (%)	69 (87)
Neoadjuvant therapy, n (%)	26 (33)
Type of surgery, n (%)
Gastrointestinal	40 (51)
Liver	14 (18)
Pancreas	23 (29)
Other	2 (3)
Operative approach, n (%)
Open	47 (60)
Robotic	24 (30)
Laparoscopic	5 (6)
Conversion to open	3 (4)
Postoperative characteristics
Drain on discharge, n (%)	7 (9)
Perioperative transfusion, n (%)	6 (8)
Postoperative complications, n (%)	27 (34)
Additional procedures, n (%)	9 (11)
Reoperation, n (%)	2 (3)
Length of stay >10 days, n (%)	8 (10)
Discharge to home, n (%)	78 (99)
Readmission ≤90 days, n (%)	17 (22)
Mortality 90 days, n (%)	0 (0)

ASA, American Society of Anesthesiologists; g/dl, grams/deciliter

Patient defined outcomes that were most commonly valued as “extremely important” were no mortality following surgery (96%), no reoperation (75%), and having a margin negative resection (73%). In contrast, those outcomes that were most commonly valued as “not important at all” or “minimally important” were receiving a blood transfusion (24%) and not having any complications at all (13%) (Figure 2).

Fig. 2

Patient defined outcomes ranked via Likert scale

Using ranking methodology, not dying following surgery was the highest ranked factor a majority of the time (89% of patients). The average rank for each listed factor is displayed in Figure 3. Following no mortality after surgery ((mean ± SD): 9.6 ± 1.5), the highest ranking factors included complete microscopic resection (8.0 ± 2.0) and no major complications (7.8 ± 1.6). Those factors that ranked consistently lower were no blood transfusion (3.4 ± 2.3), prolonged hospital stay (4.1 ± 2.5), and need to care for a drain, tube, or wound care after discharge (4.3 ± 2.5).

Fig. 3

Averaged patient defined outcomes via rank order scale

Overall, 68 patients (86%) self-reported that they experienced a TO; in contrast, only 47 patients (60%) were classified as having a TO by clinically defined criteria. Patient responses were concordant with clinically defined TO criteria 63% of the time (McNemar’s test: S=15.2, p<0.01, evidence of disagreement) (Figure 4). Of the 25 patients who self-reported TO when objectively they did not, the most common reason for not achieving a clinically defined TO was due to the occurrence a postoperative complication in 68% of cases. Of those patients who had minor complications only, 83% self-reported a TO; in comparison, 77% of patients who experienced a major complication self-reported a TO. Of those patients who experienced a minor complication only, 83% experienced a clinically defined TO; and of the patients who experienced a major complication, none experienced a clinically defined TO.

Fig. 4

Concordance between patient-reported and clinically defined measures of textbook outcome. TO, textbook outcome

The associations between clinicodemographic characteristics and patient-reported and clinically defined TO rates are shown in Table 2. While cancer characteristics including primary or recurrent cancer, stage of disease, margin of resection, operative approach, and incidence of complications did not influence patient-reported TO (all p>0.05), a decreased likelihood of a patient-reported TO was observed with incidence of perioperative transfusion (p=0.03) and readmission (p<0.01). Achievement of a clinically defined TO was associated with performance of liver surgery (p=0.04) and minimally invasive surgery (p=0.01).

Table 2

Demographic, clinical, operative, and postoperative characteristics undergoing major abdominal surgery by patient reported textbook outcomes and clinically defined textbook outcomes

Variable	Patient-reported TO	p-value	Clinically defined TO	p-value
Demographics
Age, years, n (%)
<65	43 (84)	0.54	30 (59)	1.00
≥65	25 (89)		17 (61)
Gender, n (%)
Male	29 (85)	0.86	19 (55)	0.57
Female	39 (87)		28 (62)
Race, n (%)
White	63 (86)	0.84	45 (62)	0.22
Non-White	5 (83)		2 (33)
Patient and operative characteristics
BMI, n (%)
Normal	21 (91)	0.67	14 (61)	0.96
Obese	19 (83)		14 (61)
Overweight	28 (85)		19 (58)
ASA class
2	10 (91)	0.73	6 (55)	0.90
3	56 (85)		40 (61)
4	2 (100)		1 (50)
Albumin, g/dl, n (%)
<3.5	17 (81)	0.43	11 (52)	0.44
≥3.5	51 (88)		36 (62)
Cancer diagnosis, n (%)	59 (86)	0.70	40 (58)	0.47
Neoadjuvant therapy, n (%)	22 (85)	0.79	14 (54)	0.47
Type of surgery, n (%)
Gastrointestinal	32 (80)		20 (50)
Liver	14 (100)	0.28	13 (93)	0.04
Pancreas	20 (87)		13 (57)
Other	2 (100)		1 (50)
Operative approach, n (%)
Open	39 (93)		22 (47)
Robotic	23 (96)	0.13	19 (79)	0.01
Laparoscopic	3 (60)		5 (100)
Conversion to open	3 (100)		1 (33)
Postoperative characteristics
Drain on discharge, n (%)	6 (86)	0.98	3 (43)	0.35
Perioperative transfusion, n (%)	3 (50)	0.03	-	-
Postoperative complications, n (%)	21 (78)	0.12	-	-
Additional procedures, n (%)	7 (78)	0.44	-	-
Reoperation, n (%)	1 (50)	0.26	-	-
Length of stay >10 days, n (%)	6 (75)	0.34	-	-
Discharge to home, n (%)	0 (0)	0.14	-	-
Readmission ≤90 days, n (%)	10 (59)	<0.01	-	-

ASA, American Society of Anesthesiologists; g/dl, grams/deciliter; TO, textbook outcome

Discussion

A TO describes the results of a successful surgery in which all aspects of the operation and postoperative course proceed satisfactorily (i.e., as described in the textbooks).4, 5, 12, 16, 18, 19 Compared to single perioperative metrics, the occurrence of a TO is possibly the most important outcome to surgeons, patients, hospitals, and insurers as it provides a comprehensive accounting of the overall quality of care. In this study, we characterize the patient perspective of surgical quality, clarifying the factors that are most and least important to patients in defining a TO, particularly noting the relative lack of patient prioritization on minor complications, length of stay, and postoperative blood transfusion. Herein, we also highlight the significant discordance between rates of self-reported TO and clinically defined TO.

Remarkably, there was a significant discrepancy between patient self-reported TO versus whether they experienced a clinically defined TO. Patient responses were discordant with clinically defined TO criteria more than 1/3 (37%) of the time. This inconsistency was driven largely by patient optimism; 32% of patients believed they had a TO when they actually did not meet clinically defined criteria. This may suggest there are factors that are less important to patients than providers when evaluating the outcomes of their surgery. It may also be a signal of patient expectations based on preoperative counseling of the possible adverse events that can occur after major surgery. Furthermore, cancer patients may be biased that the “cancer was removed” and less focused on short-term perioperative issues. Further research in other specialties (e.g., bariatric surgery, plastic surgery) warrants investigation to determine if this phenomenon persists.

Postoperative complications have been routinely used as criterion for exclusion for a TO, most commonly using the Clavien-Dindo classification.22, 23 Interestingly, patient’s perspectives on the importance of postoperative complications were dependent on complication severity. Specifically, patients placed a higher emphasis on avoiding major complications compared to minor complications. Indeed, many patients in our study self-reported a TO despite experiencing a minor postoperative complication. This suggests that patients are willing to tolerate minor complications but feel strongly about avoiding major complications. Given these data, defining a TO complication with a threshold of Clavien-Dindo class III or greater may better represent a more appropriate patient-centered definition. Alternatively, a more inclusive method may be to utilize the comprehensive complication index, which factors in multiple patient complications on the Clavien-Dindo classification framework into one complete index score.24

Although length of stay after surgery has been consistently included in previously published TO definitions, the current study suggests that a prolonged length of stay is of low priority to many patients in evaluating the success of their surgery. Interestingly, there were 6 patients in our study who had a prolonged hospital stay and still self-reported a TO. Previous studies have varied as to the exact definition of a prolonged hospital stay, with typical thresholds being anything greater the 50th or 75th percentile for not meeting the criteria.11, 12, 16–18, 25 For example, Merath and colleagues investigated TO rates in patients undergoing surgery for intrahepatic cholangiocarcinoma and used a length of stay >50th percentile as part of the TO definition.16 Not unexpectedly, half of the cohort was excluded from obtaining a TO for this reason, more than was excluded from all other respective factors including no complications. Thus, patient input may influence reconsideration of how length of stay is included in future TO definitions, whether labeling only severe outliers as not meeting TO criteria, or to not use length of stay as criteria at all.

Inclusion of blood transfusion in the TO definition is common which is reasonable given that transfusions are associated with perioperative morbidity and cancer recurrence.26 However, in the current study, more than half of patients (54%) did not think this factor was very important, and blood transfusion was ranked as one of the least important for defining a TO. Part of this rationale may be explained by the lack of patient knowledge on the influence a transfusion may have on his or her postoperative period, and also the minimum inconvenience patients experience with transfusion delivery. Given this discrepancy between patient opinion and existing evidence, continued inclusion of blood transfusion into definitions of TO should be carefully considered.

Composite measures are not only better than individual metrics at explaining quality of care variables such as hospital-level variation and predicting future performance,6–9 but also have significant potential to improve patient education. Patient’s understanding in informed consent is often poor and frequently inadequate.27–30 Thoughtful discussion of the likelihood of a TO and its importance with patients prior to surgery may lead to an improved understanding of expectations as well as shared decision-making, particularly if TO definitions are curated with patient input.[9, 31, 32 Furthermore, this may lead to development of individualizing expectations and understanding what is most important to an individual patient in order to help achieve a personalized TO.

These findings have important implications for future research in perioperative quality. First, as the number of studies using TO as a quality metric continues to increase, acknowledgement that physician-defined TOs differ from patient-defined metrics of TO is important. Second, adaptation of standardized TO definitions to make them more patient-centric could be considered and should be the focus of future investigations. On the other hand, while earlier studies have proceeded under the assumption that TO is a uniform definition, our findings may suggest that TO should be individualized to the patient, procedure, and other factors. In fact, assessing patient preferences preoperatively may even allow for a personalized TO metric to be used by patients, providers, and hospitals to measure the quality of patient-centered care. Finally, given occasional discordance between patient-ranked priorities and self-reported TO even when these priorities were not achieved, additional research is needed to understand the complex relationships between how individual circumstances and patient interpretation of a TO.

A limitation of the study is that the survey was designed by physicians and researchers and thus did not undergo rigorous pilot testing. In addition, the study utilized a relatively small, homogenous cohort from a single tertiary institution. As the results most closely represent the patient population at this institution, they may not be generalizable to other populations.

Omission of pain control from the current study is a limitation as patient perception of pain control could have influenced their perception of whether their outcome was “textbook” or not. As pain control has not traditionally been included in established TO metrics, it was not included in the survey design, along with other patient-reported outcomes such nausea, bowel function, functional recovery, and appetite. Given this limitation, future studies may benefit from inquiring patients perceptions on pain control and other patient-reported outcomes. The survey does not distinctly address the possible discrepancy between patient ranked factors and self-reported TO; further investigation into such currently unknown factors which may influence patient perception on whether they experienced a TO or not and additional research is necessary to further detail these relationships. Finally, patient responses may have been influenced by the timing of survey administration (i.e., after recovered from surgery and having reviewed pathology results). Overall, these results should therefore be considered exploratory in nature and used to design further prospective work utilizing larger more diverse cohorts with direct patient engagement.

Conclusions

Among patients undergoing complex abdominal surgery, there was significant discordance between patient-reported versus clinically defined measures of TO, suggesting patients value other considerations beyond traditional factors when evaluating the success of their surgery. Future studies should delineate these relationships and incorporate these factors to refine patient-centered definitions of TO.