Textbook outcome as a composite outcome measure in non-small-cell lung cancer surgery.

OBJECTIVES: Quality assessment is an important element in providing surgical cancer care. The main objective of this study was to develop a new composite measure 'textbook outcome', to evaluate and improve quality of surgical care for patients undergoing a resection for non-small-cell lung cancer (NSCLC).
METHODS: All patients undergoing an anatomical resection for NSCLC from 2012 to 2016 registered in the nationwide Dutch Lung Cancer Audit were included in an analysis to assess usefulness of a composite measure as a quality indicator. Based on expert opinion, textbook outcome was defined as having a complete resection (negative resection margins and sufficient lymph node dissection), plus no 30-day or in-hospital mortality, no reintervention in 30 days, no readmission to the intensive care unit, no prolonged hospital stay (<14 days), no hospital readmission after discharge and no major complications. The percentage of patients with a textbook outcome was calculated per hospital. Between-hospital variation in textbook outcome was analysed using case-mix adjustment models.
RESULTS: In total, 5513 patients were included in this study. Textbook outcome was achieved in 26.4% of patients. Insufficient lymph node dissection had the most substantial effect on not realizing textbook outcome. If 'sufficient lymph node dissection' was not included as a criterion, textbook outcome would be 60.7%. Case-mix adjusted textbook outcome proportions per hospitals varied between 13.2% and 37.7%.
CONCLUSIONS: In contrast to focusing on a single aspect, the composite measure textbook outcome provides insight into comprehensive performance in NSCLC surgery. It can be used to evaluate both individual hospitals and national performance and provides the opportunity to give benchmarked feedback to thoracic surgeons.

INTRODUCTION

Clinical auditing is considered an important instrument for quality assessment and improvement of quality of care [1]. National registrations provide data to evaluate the multidimensional quality of cancer care [2]. Traditionally, quality indicators mainly focused on hospital volume and 30- or 90-day mortality [3-6]. To this day, most of these quality indicators are 1-dimensional and will potentially be insufficient to monitor the multidimensional process of lung cancer surgery. Additionally, morbidity and mortality in lung cancer surgery are low; therefore, these outcome indicators have a less discriminating impact and impel less to quality improvement.

Textbook outcome as a multidimensional outcome indicator, indicating the most desirable outcome for a patient, is already being used in other fields of surgery, like colorectal, upper gastrointestinal and elective aneurysm surgery [7-9]. A textbook outcome indicator has not yet been developed for oncological lung surgery.

The main purpose of this study was to develop a new composite quality measure ‘textbook outcome’ for non-small-cell lung cancer (NSCLC) surgery and to analyse hospital variations related to this new quality indicator.

MATERIALS AND METHODS

Data source and study population

Data were derived from the Dutch Lung Cancer Audit—Surgery (DLCA-S) [10]. In 2012 the Dutch Association of Lung Surgeons and the Dutch Association of Cardiothoracic Surgery initiated the DLCA-S. This nationwide, mandatory clinical registry includes all patients undergoing surgery for NSCLC in the Netherlands. Its main purpose is to provide caregivers with feedback on quality of care and to enable a national benchmark. All patients with NSCLC clinical stage IA–IIIA who underwent a parenchymal resection [pneumonectomy, (bi)lobectomy or anatomical segmental resection] from 1 January 2012 to 31 December 2016 were included. A minimum number of items obligatory for each patient for analysis included date of birth, date of surgery, type of parenchymal resection and 30-day mortality or at hospital discharge. Patients with previous thoracic surgery on the ipsilateral side, acute surgery or neoadjuvant induction therapy were excluded. Missing items in comorbidity were considered as no comorbidity. Due to the nature of this study, no patient informed consent or approval of the medical ethical commission was necessary.

Definitions

The definition of ‘textbook outcome’, reflecting an uneventful course after parenchymal lung resection, was based on the expert opinion of the DLCA-S scientific committee. Textbook outcome is achieved when all of the following desired outcomes are realized: a complete resection (negative resection margins [11] and sufficient lymph node dissection), no 30-day or in-hospital mortality, no reintervention [reoperation (video-assisted thoracoscopic surgery/thoracotomy), bronchoscopy for atelectasis, percutaneous drainage] within 30 days after the primary operation, no readmission to the intensive care unit (ICU) or prolonged stay due to complications, no prolonged hospital stay (<14 days), no hospital readmission after discharge, no major complications [defined as prolonged air leakage (≥5 days), respiratory failure (acute respiratory distress syndrome, pulmonary oedema), myocardial infarction, thromboembolic complications, chylothorax, empyema, blood transfusion]. A sufficient lymph node dissection was defined as a dissection or a sampling of a minimum of 3 mediastinal lymph node stations (including at least the subcarinal station) and the hilar and intrapulmonary lymph nodes [12].

Analysis

The proportion of patients achieving a textbook outcome was calculated on both the national and the individual hospital level and for consecutive study years. The χ2 test was used for time trend analysis. Parameters were placed in order of clinical relevance as defined by expert opinion. If data on one of the selected outcome parameters were missing, the patient was considered not to have a textbook outcome.

The proportion of each individual outcome parameter included in the textbook outcome was calculated together with the proportion of patients in whom each subsequent parameter was realized, with the condition that all previous criteria were complied with.

To study the association between patient, tumour and treatment characteristics and textbook outcome, these characteristics were compared between the groups with and without textbook outcome using the χ2 test. Subsequently, patient and tumour covariates in the multivariable logistic regression model were selected based on known factors in the literature and on expert opinion and entered at a P-value of 0.05 using the ENTER method for variable selection in our model. Subgroups with ≤5% missing items were excluded from logistic regression analysis; patients with more than 5% missing items were analysed as a separate group. A multicollinearity test was performed to test collinearity between the covariates. Between-hospital variation in textbook outcome was demonstrated using funnel plots with 95% confidence intervals. For fair between-hospital comparisons, results were case mix adjusted by calculating observed/expected outcome ratios of the covariates from the multivariable logistic regression model [13]. The observed outcome was the number of patients with textbook outcome in a hospital, and the expected outcome was the sum of all patients’ estimated probabilities for textbook outcome. Patient probability estimates were calculated using an Enter model. An observed/expected ratio above 1 suggested a better hospital performance than average; an observed/expected ratio below 1 suggested a worse performance. For each hospital, the 95% confidence interval was calculated and plotted.

No corrections for multiple testing were done. Hospitals with fewer than 20 parenchymal lung resections a year were excluded from the analysis because they did not meet the minimal volume requirement applicable in the Netherlands to minimize statistical artefacts. A P-value <0.05 was considered statistically significant. Statistical analyses were performed in PASW Statistics version 22.0 (SPSS Inc., Chicago, IL, USA).

RESULTS

In total, 5513 patients with stage IA–IIIA NSCLC who had an anatomical parenchymal resection between 2012 and 2016 and who were registered in the DLCA-S were included for analysis. A textbook outcome was achieved in 1430 patients (26.4%) and did not significantly differ among the 4 consecutive years (P = 0.44). Thirty-four patients (0.6%) did not achieve a textbook outcome due to missing data on included outcome parameters.

Results for the overall textbook outcome result and the individual outcome parameters are presented in Fig. 1. The most substantial decrease in textbook outcome was due to insufficient lymph node dissection (55.5%), followed by readmission or prolonged stay in the ICU (13.8%) and respiratory failure (13.3%). When lymph node dissection was excluded from the definition, a textbook outcome was achieved in 60.7% of the patients. Patient, tumour and treatment characteristics of patients with and without a textbook outcome are shown in Table 1. Significant differences between the 2 groups were found in sex, age, American Society of Anesthesiologists physical status classification system, cardiac and pulmonal comorbidity, pulmonary function test results and surgical approach, but none were found in other comorbidities, clinical stage or type of resection.

Figure 1:

Diagram representing textbook outcome as composite measure of outcome parameters in patients undergoing surgery for non-small-cell lung cancer (2012–2016). Blue bar: percentage of outcome parameter; orange line: textbook outcome. ICU: intensive care unit.

Table 1:

Tumour and treatment characteristics of patients with textbook outcomes after surgery for non-small-cell lung cancer

	Textbook outcome	No textbook outcome	P-valuea
	(n = 1430), n (%)	(n = 4083), n (%)
Sex
Male	720 (50.3)	2339 (57.3)	<0.001
Female	710 (49.7)	1744 (42.7)
Age (years)
20–59	329 (23)	791 (19.4)	0.003
60–69	561 (39.2)	1573 (38.5)
70–79	472 (33)	1459 (35.7)
>80	68 (4.8)	260 (6.4)
ASA group
I–II	1120 (78.3)	3015 (73.8)	<0.001
III–IV	888 (21.7)	1068 (26.2)
Charlson score
0	500 (35)	1290 (31.6)	0.055
1	412 (28.8)	1207 (29.6)
2+	518 (36.2)	1586 (38.8)
Cardiac comorbidity
No	1087 (76)	2939 (72.0)	0.003
Yes	343 (24)	1144 (28)
Pulmonary comorbidity
No	989 (69.2)	2525 (61.8)	<0.001
Yes	441 (30.8)	1558 (38.2)
Diabetes mellitus
No	1252 (87.6)	3533 (86.5)	0.33
Yes	178 (1.4)	550 (13.5)
Neurological comorbidity
No	1223 (85.5)	3478 (85.2)	0.099
Yes	207 (14.5)	605 (14.8)
Thrombotic comorbidity
No	1382 (96.6)	3950 (96.7)	0.74
Yes	48 (3.4)	133 (3.3)
Vascular comorbidity
No	908 (63.5)	2490 (61)	0.097
Yes	522 (36.5)	1593 (39)
DLCO
>40%	700 (49)	2076 (50.8)	<0.001
<40%	14 (1)	69 (1.7)
Missing	716 (50.1)	1938 (47.5)
FEV1
>40%	456 (31.9)	1511 (37)	<0.002
<40%	8 (0.6)	40 (1)
Missing	966 (67.6)	2532 (62)
Clinical stageb
IA	586 (41.0)	1657 (40.6)	0.96
IB	319 (22.3)	934 (22.9)
IIA	233 (16.3)	647 (15.8)
IIB	182 (12.7)	513 (12.6)
IIIA	110 (7.7)	332 (8.1)
Previous thoracic surgery
No	1401 (98)	4013 (98.3)	0.44
Yes	29 (2)	70 (1.7)
Resection
Pneumonectomy	118 (8.3)	281 (6.9)	0.099
Bilobectomy	74 (5.2)	254 (6.2)
Lobectomy	1201 (84.4)	3465 (84.9)
Segmental resection	37 (2.6)	83 (2)
Approach
Video-assisted thoracoscopic surgery	1009 (70.6)	2762 (67.6)	<0.001
Thoracotomy	379 (26.5)	1156 (28.3)
Robot-assisted thoracic surgery	38 (2.7)	57 (1.4)
Missing	4 (0.3)	108 (2.6)

χ 2 test for trend.

TNM 7th edition.

ASA: American Society of Anesthesiologists; DLCO: diffusing capacity of the lung for carbon monoxide (preoperative); FEV1: forced expired volume in 1 s (preoperative).

Multivariable analysis

Independently associated factors, found in the multivariable analysis for achieving textbook outcome for patients with NSCLC, were female gender, young age, American Society of Anesthesiologists score I-II, no pulmonary comorbidity and forced expiratory volume in 1 s >40% (Table 2). No collinearity between the parameters in the multivariable analysis was found.

Table 2:

Multivariable logistic regression analysis of textbook outcome

	OR	95% confidence interval		P-value
		Lower	Upper
Sex
Male	REF			0.001
Female	1.21	1.06	1.37
Age (years)
20–59	REF			0.016
60–69	0.87	0.74	1.03
70–79	0.83	0.69	0.99
>80	0.62	0.01	0.85
ASA score
I–II	REF			0.026
III–IV	0.86	0.73	0.99
Cardiac comorbidity
No	REF			0.090
Yes	0.88	0.76	1.02
Pulmonary comorbidity
No	REF			0.001
Yes	0.75	0.65	0.86
Diabetes mellitus
No	REF
Yes	1.03	0.89	1.25	0.74
DLCO
>40%	REF			0.060
≤40%	0.56	0.32	1.02
Missing	0.89	0.75	1.05
FEV1
>40%	REF			0.039
≤40%	0.69	0.32	1.49
Missing	1.19	1.02	1.37
Previous thoracic surgery
No	REF			0.43
Yes	1.20	0.76	1.90
Clinical stage
IA	REF			0.78
IB	1.01	0.89	1.19
IIA	1.06	0.87	1.27
IIB	1.13	0.88	1.38
IIIA	1.06	0.82	1.35

ASA: American Society of Anesthesiologists; DLCO: Diffusing capacity for carbon monoxide; FEV1: forced expiratory volume in 1 s; OR: odds ratio.

Textbook outcome rates for each hospital in the Netherlands performing NSCLC resections, after case-mix adjustment, are plotted in Fig. 2A. The range of the adjusted textbook outcome of hospitals varied between 13.2% and 37.7%. One hospital with a significantly higher textbook outcome and 2 hospitals with significantly lower rates compared to the national mean textbook outcome, were identified. Figure 2B displays textbook outcome rates with sufficient lymph node dissection excluded. No significant variation was found among the hospitals.

Figure 2:

(A) Funnel plot presenting hospital variation: case-mix adjusted textbook outcome rates for individual hospitals (2012–2016). (B) Funnel plot presenting hospital variation: case-mix adjusted textbook outcome rates for individual hospitals without sufficient lymph node dissection (2012–2016). Blue dot: hospital; 95% CI: 95% confidence interval; NSCLC: non-small-cell lung cancer.

DISCUSSION

To our knowledge, this study is the first to describe a multidimensional outcome measure, textbook outcome, for the assessment of quality of care in lung cancer surgery. This composite measure, including all key surgical outcome parameters, provides insight into the overall quality of surgical care. It can be used to detect potential variations between hospitals, but most importantly to give feedback on overall performance to teams performing thoracic surgery.

Textbook outcome was realized in only 26.4% of the patients after parenchymal lung resection for a stage IA–IIIA NSCLC in our study. The main reasons why textbook outcome could not be achieved were insufficient lymph node dissection and readmission or prolonged stay in the ICU (13.8%), with a median extra admittance of 2 days. The majority of hospitals did not significantly differ in textbook outcome after case-mix adjustment, despite the range of 13.2–38.1%.

Traditionally, 1-dimensional postoperative outcome measures, such as postoperative mortality and morbidity, are considered important indicators to describe the quality of lung surgery. However, event rates of these outcomes are relatively low, which makes them less suitable to detect hospital variation. For example, overall 30-day mortality in this study is 2.1% and comparable with international described rates (2.7%, range 1.5–4.2%) [14, 15]. A composite measure like textbook outcome has a higher event rate and therefore potentially has more discriminative power to identify hospital variations in patient outcomes. Previous studies have presented alternative ways to define composite measures. Nolan and Berwick [16] noted in 2006 new ways to use composite quality indicators, by using the ‘all or nothing’ approach. This concept is also the basis of the textbook outcome concept, which reflects the desired outcome of the patient after treatment. In other words, every step or complication is equally important for the outcome of the patient. Shahian et al. [17] described a set of quality indicators for the development of a composite quality measure for cardiac surgery. Recently, 3 Dutch studies investigated textbook outcome as a composite measure for colorectal surgery, abdominal aorta aneurysm repair and upper gastrointestinal surgery [7-9]. All 3 authors described how the use of a composite measure can help evaluate hospital performance on the overall quality of surgical care and to identify between-hospital variations. For oesophageal and gastric cancer surgery in our country, textbook outcomes increased over the last 5 years from 30% to 44%, mostly by improving lymph node dissections by providing benchmarked feedback to the surgical teams [18]. Mehta et al. [19] described textbook outcome between the top 20 best performing hospitals versus the rest of the hospitals in the USA for oesophageal, liver, colorectal, pancreas and lung surgery. The total textbook outcome for lung surgery in this study was 66.2%. Mehta et al. did not include oncological outcomes as we did in our study; after excluding sufficient lymph node dissections, we achieved a textbook outcome of 60%. Nevertheless, our opinion is that radicality and sufficient lymph node dissection are beneficial for the patient and of importance in textbook outcome.

Yet, our study shows a limited discriminative effect of textbook outcome among hospitals in the Netherlands, because all hospitals have similar moderate results. By presenting hospitals results to the participating centres, comparable improvement of textbook outcomes, like those achieved for oesophageal and gastric cancer, could be pursued. Moreover, if, in an ideal situation, adequate lymph node dissection could be performed in 100% of patients in our country, there would still be substantial room for improvement, if one considers that 40% of the patients in our study also had other reasons for not being considered a textbook outcome.

Textbook outcome provides comprehensive outcome information for caregivers and patients in addition to the already frequently used 1-dimensional outcome measures. Because in the representation of a textbook outcome (Fig. 1), information on the various components is included, it can be used to direct specific improvement potential for hospitals, whereas the composite part puts the information in a broader perspective of quality evaluation. To inform patients, a composite measure may also be more understandable than separate 1-dimensional outcome measures.

Limitations

This study has several limitations. First, the DLCA-S relies on data entered by the hospitals themselves, which could potentially cause (selection) bias. However, to minimalize incomplete or unreliable data entry, several measures like internal and external quality control of the data have been taken, as reported earlier by Hoeijmakers et al. [2, 20].

Second, the textbook outcome definition was based on individual outcome indicators defined in the national lung cancer registry, DLCA-S. The definition of these individual outcome indicators can be discussed, for example, the definition of prolonged hospital stay (>14 days). As an alternative definition, the seventy fifth percentile can be used, which is 10 days, with a median of 7 days in our population. Instead of 89.3% of the patients in the 14-day definition, 79.5% of the patients will have a hospital stay of 10 days or less. This will result in a textbook outcome of 25.1%. In addition, the definition of the European Society of Thoracic Surgeons for sufficient lymph node dissection was used in our analysis. However, other lymph node dissection definitions are used, for example, the National Comprehensive Cancer Network definition, the American Cancer Society definition or that of the International Association for the Study of Lung Cancer [11, 21, 22].

In addition, the quality of the individual lymph node dissection has to be defined. Different definitions of this item make international benchmarking a challenge.

Third, the absence of long-term follow-up data in our definition of textbook outcome can be debated. Because the composite measure is meant for short-cycled feedback on performance to the hospitals participating in the DLCA-S, long-term survival was excluded. Nevertheless, textbook outcome could very well be associated with long-term survival. Recently, van der Werf et al. [23] found a positive correlation between textbook outcome and improved long-term survival for patients who had surgery for oesophageal or gastric cancer. Fourth, the definition used in our study is likely to be insufficient to comprise textbook outcome for the complete spectrum of lung cancer treatments provided to patients. Patients with neoadjuvant therapy, for example, radiotherapy and/or chemotherapy, were excluded, and adjuvant treatments were left out of our study. As of 2016, pulmonologists and radiotherapists joined the DLCA, together encompassing the whole care path of lung cancer patients in Dutch hospitals. This multidisciplinary registration will provide a more comprehensive view on the care path for lung cancer patients and potentially a more complete definition of textbook outcome in the future.

Fifth, in our current definition of textbook outcome, patient-reported outcomes (PROs) were not included because they are not yet included in the DLCA. PROs could give additional information on the outcomes of the care process as experienced and reported by patients, for example, chronic pain due to intercostal nerve damage. However, to implement PROs, an alternative textbook outcome has to be defined with the goal of measuring patient functional outcomes after lung cancer treatment. Additional research will help to determine if PROs will give more profound information to measure quality of care [24].

Finally, textbook outcome as defined in this study does not consider the ‘weight’ of individual parameters in the calculation of the final composite measure.

Future perspectives

The mean national percentage of patients in which textbook outcome was achieved (26.4%) is fairly disappointing and emphasizes the need for quality improvement in lung cancer surgery in the Netherlands. Lymph node sampling, ICU readmission and respiratory failure are particular areas that need attention. In order to improve textbook outcome at both the hospital and the national level, the results will be presented to participating hospitals using the already existing secured web-based environment called MyDLCA-S. Results of individual hospitals will be benchmarked against the national mean and outliers will be offered a visit from the DLCA auditing committee.

Moreover, targeted improvement programmes, like enhanced recovery after thoracic surgery, are developed to focus on the areas that need improvement, to decrease postoperative complications and to improve patient outcomes [25, 26].

A single international supported definition for sufficient lymph node dissection should be developed to enhance international benchmarking and quality improvement.

Future studies, with the collaboration of caregivers and patients, are needed for a more weighted textbook outcome and for more information about the association between textbook outcome and long-term results like survival.

CONCLUSION

In conclusion, textbook outcome is a new composite measure to assess quality in NSCLC surgery. It can be used to evaluate both individual hospital and national performances and between-hospital variations, with the potential to improve the overall quality of NSCLC surgery in the Netherlands.