Predictive Score of Adverse Events After Carotid Endarterectomy: The NSQIP Registry Carotid Endarterectomy Scale.

Background The goal of this study was to create a comprehensive, integer-weighted predictive scale of adverse events after carotid endarterectomy (CEA), which may augment risk stratification and patient counseling. Methods and Results The targeted carotid files from the prospective NSQIP (National Surgical Quality Improvement Program) registry (2011-2013) comprised the derivation population. Multivariable logistic regression evaluated predictors of a 30-day adverse event (stroke, myocardial infarction, or death), the effect estimates of which were used to build a weighted predictive scale that was validated using the 2014 to 2015 NSQIP registry release. A total of 10 766 and 8002 patients were included in the derivation and the validation populations, in whom 4.0% and 3.7% developed an adverse event, respectively. The NSQIP registry CEA scale included 14 variables; the highest points were allocated for insulin-dependent diabetes mellitus, high-risk cardiac physiological characteristics, admission source other than home, an emergent operation, American Society of Anesthesiologists' classification IV to V, modified Rankin Scale score ≥2, and presentation with a stroke. NSQIP registry CEA score was predictive of an adverse event (concordance=0.67), stroke or death (concordance=0.69), mortality (concordance=0.76), an extended hospitalization (concordance=0.73), and a nonroutine discharge (concordance=0.83) in the validation population, as well as among symptomatic and asymptomatic subgroups (P<0.001). In the validation population, patients with an NSQIP registry CEA scale score >8 and 17 had 30-day stroke or death rates >3% and 6%, the recommended thresholds for asymptomatic and symptomatic patients, respectively. Conclusions The NSQIP registry CEA scale predicts adverse outcomes after CEA and can risk stratify patients with both symptomatic and asymptomatic carotid stenosis using different thresholds for each population.

Clinical Perspective

What Is New?

This study uses a large patient population from the NSQIP (National Surgical Quality Improvement Program) registry to help quantify the odds of a perioperative death, infarction, or myocardial infarction on the basis of preoperative characteristics.

The scale weighs several different predictors proportional to their effect estimate in a multivariable model and was validated using patients from the more recent NSQIP registry release.

What Are the Clinical Implications?

The NSQIP registry carotid endarterectomy scale can be used to risk stratify patients being considered for carotid endarterectomy and aid in patient counseling.

Introduction

The decision analysis of pursuing carotid endarterectomy (CEA) weighs the benefit of surgery on reduced risk of cerebral infarction against the risk of developing a major perioperative adverse event, particularly stroke, death, or myocardial infarction. Preoperative characteristics impact the risk of perioperative complications; although some authors have proposed predictive models,1 including constructing predictive scales,2, 3, 4, 5, 6, 7, 8, 9 these scales have limitations that reduce their utility. An easily quantifiable, yet comprehensive, prediction score may augment risk stratification and patient counseling. The NSQIP (National Surgical Quality Improvement Program) registry is a national, multi‐institution registry that is maintained by the American College of Surgeons and enrolls patients prospectively at academic and community hospitals across the United States. The goal of the present analysis was to use the NSQIP registry targeted carotid files, which collect data on 22 variables that are pertinent to CEA,10 to construct and validate a predictive scale based on standard preoperative factors that can risk stratify both symptomatic and asymptomatic patients using the same scale, but different thresholds for each population.

Methods

Data Source and Inclusion Criteria

Data were extracted from the NSQIP registry targeted carotid files (2011–2015). All data are publicly available from the NSQIP registry.11 The NSQIP is a multi‐institutional program including federal hospitals and voluntary participation from nonfederal hospitals from varied settings across the United States. Surgical reviewers prospectively collect data with a standard protocol, and the American College of Surgeons regularly audits the data for accuracy.12, 13 The NSQIP registry follows up patients longitudinally for 30 days, and all adverse events during the hospitalization and after discharge are recorded. Studies using the deidentified NSQIP registry have been exempted as not human subject research by our institutional review board. Patients aged at least 18 years were included; these patients underwent CEA, as determined by the Current Procedural Terminology code 35301.

Predictors

Pertinent preoperative predictor variables were extracted, including patient demographics, American Society of Anesthesiologists’ physical classification designation, and functional status. Age was evaluated categorically, with the decades that approximated the median and upper quartile in the derivation population used as cutoffs. Comorbidities evaluated were smoking, hypertension requiring medication, recent congestive heart failure exacerbation (within 30 days), chronic obstructive pulmonary disease, dyspnea, diabetes mellitus, a bleeding disorder (any hypothrombotic condition other than aspirin use), and weight loss. Body habitus was classified by World Health Organization criteria as normal weight (body mass index <25 kg/m2), overweight (25 kg/m2 ≤ body mass index <30 kg/m2), class I obese (30 kg/m2 ≤ body mass index <35 kg/m2), or class II or III obese (body mass index ≥35 kg/m2). Relevant preoperative laboratory values were extracted and categorized by pertinent values: sodium (by 135 mEq/L), white blood cell count (by 4000/μL and 12 000/μL), hematocrit (by 36%), platelet count (by 150 000/μL), partial thromboplastin time (by 50 and 80 seconds), and international normalized ratio (by 1.4). Renal insufficiency was evaluated by calculating the glomerular filtration rate using the Modification of Diet in Renal Disease equation, which incorporates patient serum creatinine, age, sex, and race (stratified by 40 mL/min per 1.73 m2).14 Additional variables evaluated were revision endarterectomy (using concurrent Current Procedural Terminology code 35390) and anesthesia type.

Carotid predictors evaluated were presentation (asymptomatic, amaurosis fugax, transient ischemic attack, or stroke), preoperative modified Rankin Scale score (which is only recorded for patients who presented with a stroke), and preoperative medications. The degree of ipsilateral and contralateral carotid artery stenosis, as defined by Doppler ultrasound or angiography, is recorded by the NSQIP registry as <50%, 50% to 79%, 80% to 99%, occlusion, or not obtained. High‐risk anatomical characteristics were defined by the NSQIP registry as prior ipsilateral CEA or carotid artery stent placement, prior ipsilateral neck dissection, contralateral carotid artery occlusion, prior radiation to the neck, or contralateral laryngeal nerve injury or palsy. High‐risk physiological characteristics were defined as New York Heart Association class III/IV congestive heart failure, left ventricular ejection fraction <30%, unstable angina, or myocardial infarction within 30 days. An emergent operation includes those that are performed within 12 hours of admission. Admission type is from home versus transfer (from another short‐term care facility, the emergency department, or a nursing home).

Adverse Events

Thirty‐day complications are recorded by the NSQIP registry. A composite end point of 30‐day stroke, death, or myocardial infarction was used to build the predictive scale.15 Postoperative cerebrovascular accidents were defined by the NSQIP registry as any embolic, thrombotic, or hemorrhagic vascular accident with acute neurologic injury that persists for at least 24 hours. A myocardial infarction is indicated by new ECG findings (new left bundle branch block or ST‐segment elevation or new q waves in ≥2 leads), new elevation in troponin >3 times the upper limit of reference, or physician diagnosis. Other complications extracted were cardiac (cardiac arrest or arrhythmia requiring treatment); extubation failure; mechanical ventilation for >48 hours; symptomatic venous thromboembolism; infections (surgical site, urinary tract, pneumonia, and sepsis); packed red blood cell transfusions; and any reoperation. Carotid‐specific end points include postoperative transient ischemic attack, acute carotid artery thrombosis, and cranial nerve injury. An extended hospitalization was a hospital stay longer than the upper quartile of the derivation population. A nonroutine hospital discharge was any disposition other than to home, and readmission was any return to a short‐term care facility.

Missing Data

Data were missing on specific predictor variables, in some cases because that variable was not evaluated in individual patients (such as laboratory values that may not be obtained in all cases) and in other situations because the data were missing from the registry. For variables with missing data, the predictor was evaluated as a categorical variable, and patients with missing values were categorized as a distinct subgroup for that variable, so that known data on the predictor (patients with and without the variable) could be compared. No data were missing on adverse events.

Statistical Analysis

Statistical analyses were performed in STATA 13 (StataCorp, College Station, TX), and a probability value <0.05 defined significance. The patient population was divided into derivation (2011–2013) and validation (2014–2015) populations: the derivation population was used for the analyses that generated the predictive scale, and the validation population was used as a separate sample to evaluate the scale. Univariable logistic regression screened predictors of adverse events. Predictors with a P<0.20 in any strata were considered for inclusion in the multivariable model, which was generated with stepwise, backward selection. To optimize model parsimony, a maximum of one predictor for every 10 events was included (a standard ratio for models based on logistic regression).16 Subgroup analyses were also performed, evaluating the composite outcome stratified by symptomatic status.

Generation of the Predictive Scale

The multivariable model of adverse events was used to create the predictive scale. Point allocation was proportional to the effect size of statistically significant predictors: 3 points for an odds ratio (OR) of 1.30 to 1.49 (corresponding to a regression coefficient of 0.20–0.39), 4 points for an OR of 1.50 to 1.99 (corresponding to a regression coefficient of 0.40–0.69), and 5 points for an OR of >2.0 (corresponding to a regression coefficient of >0.70).17 In addition to the variables that were significant in the multivariable model, 2 other entry criteria into the scale were used. Two points were assigned to variables that were significant in the subgroup analyses, but not significant in the overall population. In addition, a single point was designated for variables that were available in the NSQIP registry and previously used in at least 2 other CEA risk stratification scales, but not significant in this model. Similar methods for integer‐weighted predictive scale generation have been previously used.18, 19, 20, 21, 22

Validation of the Predictive Scale

The predictive capacity of the score was evaluated in the validation population (2014–2015), as the independent variable by logistic regression with the outcome as the dependent variable. First, the composite end point was evaluated and, thereafter, other outcomes available in the NSQIP registry. Concordance statistics assessed the discrimination of all logistic regression models (ie, the ability of the model to differentiate between those who do and do not develop the outcome). Concordance statistics range between 0.50, where the model has no better prediction than chance, to 1.0, where the model has perfect discrimination.23 The calibration of models was assessed using the Hosmer‐Lemeshow test, which evaluates the null hypothesis that the observed and expected counts in each decile are equal, and a model is accepted as well calibrated if it fails to reject the null hypothesis (has a value >0.05).24

Results

Study and Validation Populations

A total of 10 766 patients were included in the derivation population (NSQIP registry 2011–2013), of whom 56.9% (n=6120) were asymptomatic, 41.2% (n=4441) were symptomatic, and 1.9% (n=205) had missing symptomatic status. The preoperative characteristics of the derivation population are shown in Table 1 and compared by the development of an adverse event in Table S1. Within the validation population (NSQIP registry 2014–2015), 8002 patients were included, in whom 54.5% were asymptomatic (n=3504), 43.8% were symptomatic (n=1695), and 1.8% (n=140) had missing symptomatic status.

Table 1

Preoperative Characteristics of the Derivation Population (NSQIP Registry 2011–2013) and Validation Population (NSQIP Registry 2014–2015)

Variable	Definition	Derivation Population, %	Validation Population, %
Age, y	18–70	44.7	46.6
	71–80	37.3	37.5
	>80	18.1	16.0
Sex	Men	60.9	61.8
	Women	39.1	38.2
Race or ethnicity	White	86.6	79.8
	Black	4.4	4.3
	Hispanic	2.4	3.2
	Asian	1.9	1.8
	Unknown	4.7	10.9
Preoperative functional status	Independent	96.5	97.3
	Dependent	3.3	2.5
	Missing	0.2	0.2
Smoking	…	26.6	27.0
Hypertension	…	85.2	82.0
COPD	…	10.0	10.1
CHF	…	1.4	1.5
Dyspnea	…	13.9	12.0
Diabetes mellitus	None	71.1	69.0
	Noninsulin	18.2	19.0
	Insulin	10.8	12.1
Bleeding disorder	…	20.8	20.0
Body habitus	Normal weight	28.3	25.7
	Overweight	38.2	38.0
	Class I obesity	21.6	22.2
	Class II/III obesity	11.0	11.8
	Missing	0.9	2.3
Weight loss	…	0.5	0.4
Preoperative sodium, mEq/L	>135	86.1	85.5
	≤135	10.1	10.2
	Missing	3.8	4.4
Preoperative GFR, mL/min per 1.73 m2	≥40	86.2	87.3
	<40	10.6	9.1
	Missing	3.2	3.6
Preoperative white blood cell count, cells/μL	4000–12 000	90.1	89.0
	>12 000	3.7	4.1
	<4000	1.8	2.1
	Missing	4.4	4.9
Preoperative hematocrit, %	>36	73.8	74.2
	≤36	23.0	21.6
	Missing	3.2	4.2
Preoperative platelet count, platelets/μL	≥150 000	85.2	85.3
	<150 000	10.3	9.9
	Missing	4.5	4.9
Preoperative PTT, s	<50	56.7	52.9
	50–80	3.7	3.5
	>80	1.3	1.0
	Missing	38.3	42.6
Preoperative INR	>1.4	71.5	66.7
	≤1.4	3.5	3.0
	Missing	25.0	30.3
Symptomatic status	Asymptomatic	56.9	54.5
	Amaurosis fugax	7.1	7.4
	TIA	15.7	16.4
	Stroke	18.4	20.0
	Missing	1.9	1.8
Preoperative modified Rankin Scale score	0–1	7.5	8.7
	2–5	7.8	7.6
	Not recorded	3.1	3.7
	Presentation other than stroke	81.6	80.0
High‐risk physiological characteristics	No	93.3	94.0
	Yes	4.9	4.7
	Missing	1.9	1.3
High‐risk anatomical characteristics	No	86.3	87.8
	Yes	11.9	10.8
	Missing	1.9	1.4
Preoperative antiplatelet agents	No	11.2	10.0
	Yes	88.2	89.7
	Missing	0.6	0.3
Preoperative statin	No	19.8	18.4
	Yes	79.6	81.3
	Missing	0.6	0.3
Preoperative β blocker	No	43.5	45.9
	Yes	55.6	53.5
	Missing	0.9	0.6
Ipsilateral carotid artery stenosis	Mild (<50%)	1.2	1.1
	Moderate (50%–79%)	28.4	30.8
	Severe (80%–99%) or occlusion	68.8	66.1
	Not obtained	1.8	2.0
Contralateral carotid artery stenosis	Mild (<50%)	49.9	51.8
	Moderate (50%–79%)	27.2	26.9
	Severe (80%–99%) or occlusion	10.5	10.4
	Not obtained	12.4	10.9
Admission type	Home	93.3	92.7
	Transfer	6.7	7.3
Emergent operation	…	2.5	2.8
ASA class	I–II	7.4	5.4
	III	76.0	74.1
	IV–V	16.5	20.3
	Missing	0.1	0.2
Revision endarterectomy	…	0.8	0.5

Glomerular filtration rate was calculated using the Modification of Diet in Renal Disease equation. Missing data for laboratory values include patients in whom such testing was not obtained. ASA physical classification designation IV is a severe systemic disease with constant threat to life and includes, but is not limited to, recent (<3 months) myocardial infarction or cardiac stent placement, severe valve dysfunction, sepsis, and renal dysfunction. ASA class V designation is a moribund patient not expected to survive without the operation. Modified Rankin Scale score of 2 to 5 indicates at least slight disability and is only recorded among patients who presented with a stroke. ASA indicates American Society of Anesthesiologists; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; GFR, glomerular filtration rate; INR, international normalized ratio; NSQIP, National Surgical Quality Improvement Program; PTT, partial thromboplastin time; TIA, transient ischemic attack.

Multivariable Logistic Regression

The total 30‐day cumulative incidence of stroke, death, or myocardial infarction in the derivation population was 4.0% (n=427; 95% CI, 3.6%–4.3%) overall and 5.0% (95% CI, 4.3%–5.6%) among symptomatic and 3.3% (95% CI, 2.8%–3.7%) among asymptomatic patients. A multivariable logistic regression model was constructed, evaluating the predictors of stroke, death, or myocardial infarction in the derivation population (Table 2). The model had a concordance statistic of 0.69 and was well calibrated.

Table 2

Multivariable Logistic Regression Model Evaluating Predictors of 30‐Day Adverse Events in the Derivation Population

Variable	Definition	OR	95% CI	P Value
Age, y	18–70	Reference	···	···
	71–80	0.99	0.79–1.25	0.95
	>80	1.34	1.03–1.75	0.03a
Race or ethnicity	White	Reference	···	···
	Black	0.71	0.42–1.20	0.21
	Hispanic	0.70	0.37–1.36	0.30
	Asian	1.41	0.89–2.50	0.23
	Unknown	0.80	0.49–1.31	0.38
Diabetes mellitus	None	Reference	···	···
	Noninsulin	1.41	1.10–1.81	0.007a
	Insulin	1.55	1.15–2.08	0.004a
Body habitus	Normal weight	Reference	···	···
	Overweight	0.95	0.75–1.22	0.70
	Class I obesity	0.91	0.68–1.22	0.53
	Class II/III obesity	0.70	0.48–1.03	0.07
	Missing	0.76	0.29–2.00	0.59
Preoperative GFR, mL/min per 1.73 m2	≥40	Reference	···	···
	<40	1.36	1.02–1.82	0.04a
	Missing	0.49	0.21–1.15	0.10
Preoperative white blood cell count, cells/μL	4000–12 000	Reference	···	···
	>12 000	0.72	0.41–1.26	0.25
	<4000	0.33	0.10–1.16	0.06
	Missing	1.54	0.67–3.53	0.31
Preoperative hematocrit, %	>36	Reference	···	···
	≤36	1.22	0.97–1.53	0.09
	Missing	1.15	0.42–3.19	0.79
Symptomatic status	Asymptomatic	Reference	···	···
	Amaurosis fugax	0.96	0.61–1.50	0.84
	TIA	1.37	1.04–1.80	0.02a
	Stroke	2.38	1.55–3.67	<0.001a
	Missing	1.00	0.46–2.19	0.99
Preoperative modified Rankin Scale score	0–1	Reference	···	···
	2–5	2.16	1.33–3.50	0.002a
	Not recorded or presentation other than stroke	2.57	1.45–4.54	0.001a
High‐risk physiological characteristics	…	1.93	1.38–2.69	<0.001a
	Missing	2.01	0.82–4.93	0.13
High‐risk anatomical characteristics		1.35	1.02–1.78	0.03a
	Missing	0.88	0.34–2.29	0.79
Preoperative β blocker		1.05	0.85–1.29	0.65
	Missing	0.40	0.09–1.69	0.21
Contralateral carotid artery stenosis	Mild (<50%)	Reference	···	···
	Moderate (50%–79%)	1.22	0.96–1.54	0.10
	Severe (80%–99%) or occlusion	1.38	1.01–1.90	0.04a
	Not obtained	1.41	1.04–1.92	0.03a
Admission type	Transfer	1.54	1.12–2.11	0.008a
Emergent operation	…	1.70	1.06–2.71	0.03a
ASA class	I–II	Reference	···	···
	III	1.13	0.71–1.81	0.60
	IV–V	1.94	1.18–3.18	0.009a
Concordance statistic	…	0.69
Hosmer‐Lemeshow test	…	0.15

ASA indicates American Society of Anesthesiologists; GFR, glomerular filtration rate; OR, odds ratio; TIA, transient ischemic attack.

Statistically significant difference.

Additional analyses evaluated predictors of an adverse event, stratified by symptomatic status (Table S2). Statistically significant independent predictors among symptomatic patients (in descending effect size) were high‐risk physiological characteristics (P=0.001), modified Rankin Scale score of 2 to 5 (P=0.004), hospital transfer (P=0.006), hypertension (P=0.03), anemia (P=0.002), and aged >80 years (P=0.04); in asymptomatic patients, predictors were hospital transfer (P=0.005), American Society of Anesthesiologists’ class 4 to 5 designation (P=0.03), contralateral carotid imaging not obtained (P=0.02), insulin‐dependent diabetes mellitus (P=0.02), and renal insufficiency (P=0.02). The concordance statistics of the models were 0.73 for symptomatic and 0.68 for asymptomatic patients, and both models were well calibrated.

NSQIP Registry CEA Scale

The ORs from the multivariable logistic regression model of a major adverse event were used to construct the NSQIP registry CEA scale; moreover, chronic obstructive pulmonary disease was included and designated a single point because of its use in prior risk stratification scales, and anemia as well as hypertension were assigned 2 points given their significance in the subgroup analysis of symptomatic patients. The components of the scale, as well as their prevalence in the derivation and validation populations, are listed in Table 3.

Table 3

Components of the NSQIP Registry CEA Scale Score

Points	Variable	Stratification	Prevalence, %
			Derivation Population (2011–2013)	Validation Population (2014–2015)
1	Comorbidity	Chronic obstructive pulmonary disease	10.0	10.1
2	Comorbidity	Hypertension	85.2	82.0
2	Laboratory value	Anemia (hematocrit <36%)	23.0	21.6
3	Patient age	>80 y	18.1	16.0
3	Comorbidity	Non–insulin‐dependent diabetes mellitus	18.2	19.0
3	Laboratory value	Renal insufficiency (GFR <40 mL/min per 1.73 m2)	10.6	9.1
3	Symptomatic	Presentation with a transient ischemic attack	15.7	16.4
3	High‐risk features	High‐risk anatomical characteristics	11.9	10.8
3	Contralateral carotid artery stenosis	High‐grade stenosis (80%–99%), occlusion, or imaging not obtained	22.9	21.3
4	Comorbidity	Insulin‐dependent diabetes mellitus	10.8	12.1
4	High‐risk features	High‐risk physiological characteristics	4.9	4.7
4	Admission type	Hospital transfer, emergency department, or nursing home	6.7	7.2
4	Case urgency	Emergent	2.5	2.8
4	ASA classification	IV–V	16.5	20.3
5	Symptomatic	Presentation with a stroke	18.4	20.0
5	Modified Rankin Scale	Score 2–5	7.8	7.6
47	Maximum potential score		…	…

High‐risk anatomical characteristics were defined by the NSQIP registry as prior ipsilateral CEA or carotid artery stent placement, prior ipsilateral neck dissection, contralateral carotid artery occlusion, prior radiation to the neck, or contralateral laryngeal nerve injury or palsy. High‐risk physiological characteristics were defined as New York Heart Association class III/IV congestive heart failure, left ventricular ejection fraction <30%, unstable angina, or myocardial infarction within 30 days. ASA physical classification designation IV is a severe systemic disease with constant threat to life and includes, but is not limited to, recent (<3 months) myocardial infarction or cardiac stent placement, severe valve dysfunction, sepsis, and renal dysfunction. ASA class V designation is a moribund patient not expected to survive without the operation. Modified Rankin Scale score of 2 to 5 indicates at least slight disability and is only recorded among patients who presented with a stroke. ASA indicates American Society of Anesthesiologists; CEA, carotid endarterectomy; GFR, glomerular filtration rate; NSQIP, National Surgical Quality Improvement Program.

Validation of the NSQIP Registry CEA Scale

In the validation population (2014–2015; n=8002), the total 30‐day cumulative incidence of stroke, death, or myocardial infarction was 3.7% (n=297; 95% CI, 3.3%–4.1%). Greater NSQIP registry CEA scale score was predictive of the composite end point, each end point individually, stroke or death, any complication, an extended hospitalization, a nonroutine hospital discharge, and an unplanned readmission in the validation population (Table 4, Figure). However, the NSQIP registry CEA score was not predictive of purely technical complications, which are not expected to vary by preoperative characteristics, including cranial nerve injury or an acute postoperative carotid artery thrombosis (OR, 2.00; 95% CI, 0.28–14.6; P=0.49; concordance, 0.50).

Table 4

Predictive Capacity of the NSQIP Registry CEA Scale in the Validation Population, as Well as Stratified by Symptomatic Status

Outcome	Population	30‐d Rate (95% CI), %	Concordance	OR	95% CI	P Value	HL
Stroke, death, or myocardial infarction	Total	3.7 (3.3–4.1)	0.67	1.10	1.08–1.11	<0.001a	0.22
	Symptomatic	4.8 (4.1–5.5)	0.63	1.08	1.05–1.10	<0.001a	0.21
	Asymptomatic	2.8 (2.3–3.3)	0.66	1.14	1.10–1.08	<0.001a	0.20
Stroke or death	Total	2.8 (2.5–3.2)	0.69	1.11	1.09–1.13	<0.001a	0.16
	Symptomatic	4.2 (3.5–4.8)	0.64	1.08	1.05–1.11	<0.001a	0.69
	Asymptomatic	1.7 (1.4–2.1)	0.67	1.15	1.10–1.20	<0.001a	0.15
Death	Total	0.8 (0.6–1.0)	0.76	1.15	1.11–1.19	<0.001a	0.92
	Symptomatic	1.1 (0.8–1.5)	0.75	1.14	1.09–1.19	0.001a	0.99
	Asymptomatic	0.6 (0.3–0.8)	0.74	1.21	1.12–1.30	<0.001a	0.74
Stroke	Total	2.2 (1.9–2.5)	0.68	1.09	1.07–1.12	<0.001a	0.14
	Symptomatic	3.4 (2.8–4.0)	0.61	1.06	1.03–1.09	<0.001a	0.34
	Asymptomatic	1.3 (1.0–1.6)	0.64	1.12	1.06–1.19	<0.001a	0.35
Myocardial infarction	Total	1.4 (1.1–1.6)	0.62	1.07	1.04–1.10	<0.001a	0.59
	Symptomatic	1.3 (0.9–1.6)	0.62	1.08	1.03–1.13	0.001a	0.58
	Asymptomatic	1.4 (1.1–1.9)	0.65	1.13	1.07–1.19	<0.001a	0.73
Cranial nerve deficit	Total	3.1 (2.7–3.5)	0.51	1.01	0.98–1.03	0.56	0.17
	Symptomatic	2.9 (2.4–3.5)	0.50	1.00	0.97–1.04	0.89	0.10
	Asymptomatic	3.1 (2.6–3.7)	0.53	1.03	0.99–1.07	0.16	0.35
Any complication	Total	10.3 (9.7–11.0)	0.64	1.09	1.07–1.10	<0.001a	0.85
	Symptomatic	12.8 (11.7–13.9)	0.62	1.07	1.05–1.09	<0.001a	0.99
	Asymptomatic	8.4 (7.5–9.2)	0.63	1.12	1.10–1.15	<0.001a	0.70
Extended hospital stay	Total	27.7 (26.8–28.7)	0.73	1.18	1.16–1.19	<0.001a	0.06
	Symptomatic	44.6 (42.9–46.2)	0.69	1.13	1.12–1.15	<0.001a	0.33
	Asymptomatic	14.4 (13.4–15.4)	0.64	1.13	1.11–1.15	<0.001a	0.14
Nonroutine discharge	Validation	6.6 (6.1–7.2)	0.83	1.23	1.21–1.35	<0.001a	0.09
	Symptomatic	12.2 (11.1–13.2)	0.80	1.21	1.18–1.23	<0.001a	0.001
	Asymptomatic	3.3 (2.7–3.9)	0.71	1.18	1.14–1.23	<0.001a	0.52
Unplanned readmission	Total	5.5 (5.0–6.0)	0.62	1.07	1.05–1.08	<0.001a	0.09
	Symptomatic	6.2 (5.4–7.0)	0.58	1.04	1.02–1.07	<0.001a	0.81
	Asymptomatic	3.3 (2.7–3.9)	0.65	1.14	1.10–1.17	<0.001a	0.26

An extended hospital stay was defined as longer than the upper quartile of the population (of at least 3 days). A nonroutine hospital discharge is any discharge other than to home. CEA indicates carotid endarterectomy; HL, Hosmer‐Lemeshow test; NSQIP, National Surgical Quality Improvement Program; OR, odds ratio.

Statistically significant difference.

Figure 1

Variations in the rates (and associated SEMs) of 30‐day adverse events (stroke, death, or myocardial infarction; A), mortality (B), an extended hospitalization (of at least 3 days; C), and nonroutine hospital discharge (D) in the derivation and validation populations. CEA indicates carotid endarterectomy; NSQIP, National Surgical Quality Improvement Program.

The NSQIP registry CEA scale score was predictive of the composite end point and other expected outcomes in both symptomatic and asymptomatic patients. An additional subgroup analysis evaluated the outcomes of asymptomatic patients on preoperative antiplatelet agents and statin (n=3340/4358, 76.6%): NSQIP registry CEA scale score was also predictive of postoperative outcomes (except cranial nerve injury; Table S3). Thereafter, risk stratification categories were designated on the basis of the point allocations from the NSQIP registry CEA scale, and the 30‐day rates of adverse events are presented in Table 5. Patients with a very‐high‐risk designation had a 30‐day rate of stroke or death of >6.0%, and patients with at least a medium‐risk designation had a stroke or death rate of >3.0%, the respective recommended thresholds for symptomatic and asymptomatic patients, respectively.

Table 5

Total 30‐Day Rates of Complications in the Validation Population, Stratified by the NSQIP Registry CEA Scale Divisions

Outcome	Very Low Risk	Low Risk	Medium Risk	High Risk	Very High Risk
NSQIP registry CEA scale points	0–5	6–8	9–12	13–16	≥17
Overall prevalence, %	38.9	20.1	20.6	11.9	8.5
Prevalence: symptomatic patients, %	19.9	17.3	25.7	19.6	17.6
Prevalence: asymptomatic patients, %	54.2	22.3	16.5	5.7	1.4
Stroke, death, or MI, %	1.6	3.4	4.2	7.2	8.3
Stroke or death, %	1.0	2.5	3.2	5.8	6.9
Death, %	0.2	0.5	0.6	2.1	3.1
Stroke, %	0.8	2.1	2.7	4.4	4.6
MI, %	0.8	1.2	1.6	2.5	2.4
Any complication, %	6.0	9.0	11.8	15.5	22.6
Extended hospitalization, %	12.8	22.0	32.9	47.9	69.3
Nonroutine discharge, %	1.0	2.7	6.0	14.7	32.3
Unplanned readmission, %	3.2	4.8	7.2	8.8	8.7

CEA indicates carotid endarterectomy; MI, myocardial infarction; NSQIP, National Surgical Quality Improvement Program.

Discussion

The NSQIP registry carotid target files provide a source of prospectively collected data from across the United States, with several pertinent variables and end points related to carotid artery revascularization. Although some studies have used the NSQIP registry before the release of the CEA target files to model outcomes,1, 7, 25, 26, 27 they were not able to use the carotid‐specific variables, many of which emerged as predictors in the NSQIP registry CEA scale. The sole publication to use the NSQIP registry carotid files to examine adverse events only analyzed a single year of the release and, thereby, only a few pertinent predictors were identified and no predictive scale was proposed.10

Risk stratification of outcomes after CEA is complex, as there are many pertinent preoperative and operative factors. Although some have published models quantifying this risk,1 including predictive scales,2, 3, 4, 5, 6, 7, 8, 9 such models have key limitations: many did not include different derivation and validation populations, and those that did include a unique validation population were restricted to asymptomatic patients,4 accrued patients from a single institution,6 or provided significant weight to limited variables (admission from a nursing home or urgent procedure).9 A validated model of outcomes after CEA that is simultaneously comprehensive, incorporating known predictors, yet also based on a straightforward calculation, would aid in risk stratification.

The NSQIP registry CEA scale underscores the importance of 14 preoperative characteristics in predicting postoperative outcomes. There are many advantages to the NSQIP registry CEA scale. First, the scale is validated and is predictive of several key postoperative outcomes. The use of the more recent years as the validation population highlights the current applicability of the score, and a separate validation population increases the external validity compared with models constructed with boostrapping.28 In the validation population, the scale was not only predictive of 30‐day stroke, death, or myocardial infarction, but it was also predictive of each outcome individually, any complication, an extended hospitalization, a nonroutine discharge, and readmission. Notably, the NSQIP registry CEA scale had greater discrimination in the validation population of death (concordance=0.76), stroke or death (concordance=0.69), an extended hospitalization (concordance=0.73), and a nonroutine discharge (concordance=0.83), compared with the composite end point (concordance=0.67) on which it was constructed. However, the scale was not predictive of purely technical outcomes that were expected to depend only on operative factors, including cranial nerve injury and acute carotid artery thrombosis, highlighting its selective prediction of end points that differ by preoperative characteristics.

In addition, the NSQIP registry CEA scale can be used to risk stratify both symptomatic and asymptomatic patients, as it was predictive of the composite end point and pertinent outcomes in both groups. This differs from some predictive scales of CEA that were restricted by symptomatic status,4 making it more parsimonious and, thereby, increasing its applicability. Although the prevalence rates of high‐ and very‐high‐risk designations are expected to be greater in symptomatic patients, given the inclusion of symptomatic status, emergent surgery, and modified Rankin Scale score in the NSQIP registry CEA scale, different thresholds of adverse events can be used for acceptable risk profile for symptomatic and asymptomatic patients. Furthermore, as maximal medical therapy is the first‐line treatment for asymptomatic stenosis,29 a subgroup analysis also evaluated asymptomatic patients taking preoperative antiplatelet and statin medications, in whom the NSQIP registry CEA scale was also predictive of outcomes.

The NSQIP registry CEA scale includes, in a single scale, many preoperative variables that are known to impact perioperative risk (including those used in other predictive scales), and weighs these factors accordingly. The components of the scale are standard preoperative characteristics that are known when the decision to pursue surgery is made, allowing the scale to risk stratify patients preoperatively. Most components from prior scales that are available in the NSQIP registry were statistically significant independent predictors of adverse events and, thereby, included in the scale; given that chronic obstructive pulmonary disease has been included in 2 prior models,1, 6 but was not a significant predictor in this model, a single point was assigned for this variable. Moreover, some variables used in prior scales are included among composite predictors that are components of the NSQIP registry CEA scale (eg, congestive heart failure,4, 8, 9 which is included among high‐risk physiological characteristics; prior neck irradiation,6 which is included among high‐risk anatomical characteristics; and cardiac valvular disease,4 which is included among American Society of Anesthesiologists’ classification 4 designation).

In addition, the scale is based on an integer‐weighting system of the effect size from multivariable logistic regression, creating a scale that can be calculated in a clinical setting with appropriate weighting without the need of a formula. Although the estimation of the effect size does lose some information (compared with a full regression equation using β coefficients), the discriminations of the linearized scale (concordance=0.67) and the entire model (concordance=0.69) were similar for the composite end point, thereby favoring the simpler model. Finally, the national enrollment of patients in the NSQIP registry increases the generalizability of results compared with single‐center or regional studies.

Nevertheless, there are several limitations of the NSQIP registry CEA scale. First, only 30‐day outcomes are available; and long‐term morbidity could not be ascertained. In addition, some pertinent variables are not included in the current NSQIP registry algorithm, including laterality of stenosis, preoperative atrial fibrillation, peripheral vascular disease, and cardiac valvular disease. Moreover, although plaque morphological characteristics are important in risk stratification, particularly of asymptomatic patients, only the severity of stenosis is recorded by the NSQIP registry. Furthermore, the deidentified NSQIP registry participant use files do not provide surgeon‐ or hospital‐level data; and differences in outcomes by surgical experience or volume could not be evaluated.

Moreover, the concordance statistic of the overall model (0.69) indicates moderate discrimination. This is not surprising as the goal was to evaluate the association of preoperative factors with adverse outcomes after CEA; however, operative and surgeon characteristics will also have a significant impact on postoperative outcomes.8 The concordance statistics are comparable with previously published risk stratification scores of CEA (ranging from 0.62 to 0.74) in their respective study populations. Finally, some have argued that the concordance statistic has limited utility in surgical outcomes research, and may have greater association with case mix and homogeneity rather than model discrimination.30

The implications of the NSQIP registry CEA scale merit further comment. As an epidemiologic tool, the scale may identify baseline differences in risk profile,18 and can be used in future observational studies and trials to quantify, compare, and adjust for perioperative risk. For clinical risk stratification, the score may be of the greatest utility among patients in the very‐high‐risk category (who had a score of at least 17 and comprised 8.5% of the validation population), where the 30‐day rate of stroke or death was 6.9% in the validation population. Guidelines recommend pursuing CEA in symptomatic patients when the risk of stroke or death is <6.0%,31 and a 3% target for stroke or death in asymptomatic patients has been suggested by the American Association of Neurology.32 Therefore, symptomatic patients in the very‐high‐risk category (17.6% of the validation population) and asymptomatic patients in the medium‐, high‐, and very‐high‐risk categories (23.6% of the validation population) should be counseled about their greater odds of adverse events; and the decision to pursue surgery should be determined on the basis of each patient and his or her physicians.

Future research will be needed to evaluate (and potentially modify) the NSQIP registry CEA scale by examining its discrimination and comparing it with other prior published scales in a validation population other than the NSQIP registry. Another potential direction would be to evaluate the predictive capability of the NSQIP registry CEA score after accounting for differences in surgeon volume, experience, and prior rates of adverse outcomes. Nevertheless, such calculations can be somewhat cumbersome in a clinical setting; and ultimate implementation of a computer program or an electronic application may optimize user efficiency.33 Moreover, additional studies that weigh the NSQIP registry CEA scale score against an individual's predicted probability of infarction from carotid stenosis, particularly among asymptomatic patients, may optimally use the NSQIP registry CEA scale for risk stratification.

Conclusions

In the present analysis, the NSQIP registry CEA scale was developed using preoperative characteristics to provide a quantifiable metric to risk stratify both symptomatic and asymptomatic patients. The scale was constructed on 14 preoperative characteristics and was predictive of the composite end point of a major adverse event (stroke, death, or myocardial infarction), an extended hospitalization, a nonroutine hospital discharge, and an unplanned readmission in the validation population, among both symptomatic and asymptomatic patients. Patients with an NSQIP registry CEA scale score of at least 17 (8.5% of patients) had a 6.9% rate of stroke or death in the validation population, highlighting patients who have the greatest odds of a perioperative adverse event.

Disclosures

Dr Aziz‐Sultan is a proctor for Covidien and Codman. Dr Gormley is a proctor for Codman. The remaining authors have no disclosures to report.

Table S1. The Demographics and Preoperative Characteristics of the Derivation Population (NSQIP 2011–2013) and Univariable Logistic Regression Evaluating 30‐Day Stroke, Death or Myocardial Infarction

Table S2. Multivariable Regression Evaluating the Predictors of Stroke, Death, or Myocardial Infarction in the Derivation Population, Stratified by Symptomatic Status

Table S3. Prediction of the NSQIP CEA Scale in the Validation Population Among Asymptomatic Patients on Antiplatelet Agents and Statin Preoperatively (n=3340)

Click here for additional data file.