A Comparison of Common Plastic Surgery Operations Using the NSQIP and TOPS Databases.

Both the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) and the American Society of Plastic Surgeons Tracking Operations and Outcomes for Plastic Surgeons (TOPS) databases track 30-day outcomes.
METHODS: Using the 2008-2016 TOPS and NSQIP databases, we compared patient characteristics and postoperative outcomes for 5 common plastic surgery procedures. A weighted TOPS population was used to mirror the NSQIP population in clinical and demographic characteristics to compare postoperative outcomes.
RESULTS: We identified 154,181 cases. Compared with NSQIP patients, TOPS patients were more likely to be younger (47.9 versus 50.0 years), have American Society of Anesthesiologists class I-II (92.1% versus 74.6%), be outpatient (66.0% versus 49.3%), and be smokers (18.7% versus 11.7%). TOPS had extensive missing data: body mass index (40.6%), American Society of Anesthesiologists class (34.9%), diabetes (39.3%), and smoking status (37.2%). NSQIP was missing <1% of all shared categories except race (15.6%). The entire TOPS cohort versus only TOPS patients without missing data had higher rates of dehiscence (5.1% versus 3.5%) and infection (2.1% versus 1.7%). TOPS versus NSQIP patients had higher dehiscence rates (5.1% versus 1.0%) but lower rates of return to the operating room (3.1% versus 6.6%), infection (2.1% versus 3.0%), and medical complications (0.3% versus 2.2%). Nonweighted and weighted TOPS cohorts had similar 30-day outcomes.
CONCLUSIONS: NSQIP and TOPS populations are different in characteristics and outcomes, likely due to differences in collection methodology and the types physicians using the databases. The strengths of each dataset can be used together for research and quality improvement.

INTRODUCTION

National databases collect and track patient information such as demographic data, administrative and cost data, medical history, health status, test results, procedures performed, and health outcomes in a given healthcare system.[1-3] Analysis of these data is used to direct policy and improve quality of care and patient outcomes.[4-7] The content and quality of databases vary, making it critical for physicians and researchers to understand the populations and specific data that a database contains. When used appropriately, databases have the power to influence positive change across healthcare systems.[1,4,8-11]

National databases are commonly used for surgical outcomes research. A large body of work in plastic surgery has used the American College of Surgeons—National Surgical Quality Improvement Program (NSQIP) database.[12-18] An alternative database, the American Society of Plastic Surgeons (ASPS)—Tracking Operations and Outcomes for Plastic Surgeons (TOPS) has recently been considered for outcomes research specifically in plastic surgery but has a less well-understood patient population.[19,20]

There are several key differences between the NSQIP and TOPS. First, cases are sampled randomly for the NSQIP, whereas ASPS members have the ability to choose the cases they contribute to TOPS. Entry of certain demographic information into TOPS is not required. Lastly, NSQIP data are abstracted by a third party, whereas in TOPS, data are entered by the surgeons themselves, which can then lead to bias.

To better understand the differences between NSQIP and TOPS, we compared data from each of the databases for a variety of plastic surgery procedures. The goal of this study was to compare common plastic surgery procedures available in both datasets for differences in patient characteristics, missing data, and outcomes with the intent to inform future researchers on the characteristics of the populations within each dataset, the differences in complication rate reporting, and how to use each database to produce reliable and valid evidence.

METHODS

Data and Study Cohort

Our comparison of TOPS and NSQIP involved patients 18–89 years of age undergoing 1 of 5 procedures between 2008 and 2016. These procedures were identified using Current Procedural Terminology (CPT) codes and include breast reduction (19318), prosthetic breast reconstruction (19357, 19342), autologous breast reconstruction (19361, 19364, 19367), free-flap procedures (15756, 15757, 15758, 20969, 20955), and pressure ulcer repair (15922, 15934, 15936, 15944, 15946, 15952, 15956). From each dataset, we collected a case for each surgery type if one of the above-mentioned CPT codes was recorded in any CPT field.

The NSQIP is a validated and nationally representative database designed for 30-day surgical outcomes research across a broad spectrum of surgical specialties. In brief, NSQIP data are currently collected at over 600 hospitals in 49 states across the United States.[21] At participating sites, surgically trained nurses collect data from patient medical charts on randomly assigned cases from all surgical specialties. The data recorded include case type, hospital characteristics, patient preoperative demographic and clinical characteristics, intraoperative characteristics, and 30-day surgical outcomes. Importantly, the NSQIP can be viewed as a nationally representative random sample of surgical cases. However, the NSQIP lacks complication reporting for many specific outcomes of interest for plastic surgeons, such as loss of graft, flap, or prosthesis.[12] The TOPS database is designed to track 30-day surgical outcomes for plastic surgery patients, specifically. Implemented initially in 2002, TOPS is contributed to by over 700 ASPS member surgeons across the United States. Surgeons enter data directly for the TOPS dataset using an electronic data capture interface, where they are prompted to enter case type and 30-day surgical outcomes, as well as patient demographic and clinical information.[22]

The 2 primary goals of our analysis were to compare patient characteristics and surgical complication rates across the 2 datasets. The 2 datasets both collect the following patient characteristics: age, body mass index (BMI), smoking status, diabetes status, inpatient/outpatient surgery status, and American Society of Anesthesiologists (ASA) classification score. Regarding the surgical outcomes reported by both datasets, we compared medical complications (cardiac arrest, myocardial infarction, stroke/cerebrovascular accident (CVA) occurrence, renal failure, organ space infection, systemic shock, systemic sepsis, pneumonia, reintubation, or urinary infection), wound dehiscence, mortality, venous thromboembolism (deep venous thrombosis [DVT] or pulmonary embolism), return to operating room (OR), and surgical infection (progressive renal insufficiency, superficial surgical site infection, and wound infection). We restricted our cohort to those with documented BMI >10 and <100. Our study protocol was reviewed by the University of Utah Institutional Review Board and was given an exempt status.

Statistical Analyses

Patient characteristics and 30-day outcomes were compared across the 2 datasets. In each, descriptive statistics are provided, where we give mean and SD for continuous variables and counts and percentages for categorical variables. Hypothesis-driven tests are provided for most comparisons. χ2 tests or exact tests were used for categorical variable comparisons, where appropriate. T tests were used to compare continuous covariates. In analyses with weighted populations, we used survey-weighted χ2 tests or survey-weighted exact tests.

In our first analysis, we compare patient characteristics across datasets within each procedure category for all available information within a given variable. As has been discussed in previous research, TOPS suffers from extensive missing data.[20] With the goal of generalizing TOPS surgical complications to the population represented by NSQIP, we sought to eliminate patients with any field of missing demographic or preoperative clinical information from the analysis. To understand bias induced by including only complete TOPS cases, we summarized the extent of missing data across the datasets. We then compared complications in the entire TOPS cohort versus the TOPS cohort with complete data and compared complications in the complete TOPS dataset to the complete NSQIP patients. Within the complete TOPS cases, we sought to compare complication rates in a weighted TOPS sample. The weights were computed to generate a pseudo-population of TOPS patients, where TOPS and NSQIP are as close as possible to being identical in distribution with respect to all patient characteristics discussed above. We chose to use inverse probability of selection weights (IPSWs), which are proposed in the literature, where the intent is to generalize results from randomized controlled trials to a target population.[23] We developed a model for the IPSW within each surgical category because we felt each surgical category represented potentially unique patient populations. In each case, the selection model involved a multivariable logistic regression on all preoperative patient variables shared across the datasets (each variable in Table 1), where interaction and higher order terms were included if the resultant model improved the balance in covariates across datasets. These variables were included because we felt each variable has the potential to influence complications and is shown to differ across the datasets in distribution. Balance was assessed in each case using a weighted absolute standardized mean difference. Once IPSWs were generated for the TOPS dataset, complications in the weighted TOPS dataset were compared with those in the unweighted NSQIP. All analyses were performed using RStudio Version 1.1.383 (2009–2017 RStudio, Inc., Boston, Mass.).

Table 1.

Crude Data Comparisons of Preoperative Patient Characteristics

	Overall			Breast Reductions			Prosthetic Reconstruction			Autologous Reconstruction			Free Flap			Pressure Ulcer Repair
	NSQIP	TOPS	P	NSQIP	TOPS	P	NSQIP	TOPS	P	NSQIP	TOPS	P	NSQIP	TOPS	P	NSQIP	TOPS	P
N	94,823	59,358		26,012	29,746		48,219	22,749		14,289	4787		5085	1017		1218	1059
Age, mean (SD)	50.00 (12.69)	47.85 (13.31)	<0.001	44.96 (14.49)	43.80 (14.07)	<0.001	51.12 (10.96)	51.87 (10.85)	<0.001	51.90 (9.89)	51.59 (9.80)	0.062	58.79 (14.72)	52.44 (16.85)	<0.001	54.77 (16.31)	51.83 (16.18)	<0.001
ASA class, n (%)			<0.001			<0.001			<0.001			<0.001			<0.001			<0.001
1	11,198 (11.8)	14,613 (37.8)		6275 (24.1)	10,439 (50.3)		4097 (8.5)	3508 (24.4)		706 (4.9)	483 (20.1)		117 (2.3)	92 (19.9)		3 (0.2)	91 (14.1)
2	59,467 (62.8)	20,957 (54.3)		15,688 (60.4)	9304 (44.8)		33,233 (69.0)	9537 (66.4)		9008 (63.1)	1682 (70.0)		1412 (27.8)	217 (47.0)		126 (10.4)	217 (33.5)
3	23,194 (24.5)	2954 (7.6)		3950 (15.2)	999 (4.8)		10,718 (22.2)	1283 (8.9)		4517 (31.6)	236 (9.8)		3204 (63.1)	142 (30.7)		805 (66.3)	294 (45.4)
4	863 (0.9)	86 (0.2)		72 (0.3)	13 (0.1)		123 (0.3)	17 (0.1)		46 (0.3)	3 (0.1)		342 (6.7)	11 (2.4)		280 (23.1)	42 (6.5)
5	2 (0.0)	15 (0.0)		1 (0.0)	1 (0.0)		0 (0.0)	11 (0.1)		0 (0.0)	0 (0.0)		1 (0.0)	0 (0.0)		0 (0.0)	3 (0.5)
BMI, mean (SD)	28.68 (6.57)	28.72 (6.12)	0.334	31.25 (6.44)	30.11 (5.99)	<0.001	27.40 (6.36)	26.85 (5.85)	<0.001	29.12 (5.97)	28.47 (5.53)	<0.001	26.91 (6.46)	27.67 (6.35)	0.041	27.29 (7.68)	27.58 (7.64)	0.558
Diabetes (preop), n (%)	5989 (6.3)	1459 (4.0)	<0.001	1483 (5.7)	639 (3.3)	<0.001	2666 (5.5)	581 (4.2)	<0.001	937 (6.6)	118 (5.1)	0.008	613 (12.1)	28 (7.7)	0.015	290 (23.8)	93 (18.3)	0.015
Outpatient, n (%)	46,721 (49.3)	39,139 (66.0)	<0.001	22,595 (86.9)	24,841 (83.5)	<0.001	22,945 (47.6)	13,451 (59.3)	<0.001	910 (6.4)	614 (12.9)	<0.001	144 (2.8)	58 (5.7)	<0.001	127 (10.4)	175 (16.5)	<0.001
Male sex, n (%)	4514 (4.8)	2001 (3.4)	<0.001	330 (1.3)	468 (1.6)	0.004	95 (0.2)	126 (0.6)	<0.001	49 (0.3)	29 (0.6)	0.02	3237 (63.7)	647 (63.6)	0.997	803 (65.9)	731 (69.0)	0.126
Smoker status, n (%)	11,082 (11.7)	6976 (18.7)	<0.001	2489 (9.6)	3014 (15.2)	<0.001	5550 (11.5)	3081 (21.9)	<0.001	1372 (9.6)	494 (20.8)	<0.001	1407 (27.7)	194 (47.5)	<0.001	264 (21.7)	193 (33.6)	<0.001
Race, n (%)			<0.001			<0.001			<0.001			<0.001			<0.001			<0.001
White	64,563 (81.9)	43,089 (83.0)		13,044 (71.5)	20,918 (79.3)		36,931 (86.3)	18,054 (88.0)		10,022 (80.7)	2910 (81.8)		3766 (87.5)	605 (84.7)		800 (73.6)	602 (78.1)
Black	11,124 (14.1)	5464 (10.5)		4853 (26.6)	3782 (14.3)		3819 (8.9)	1183 (5.8)		1834 (14.8)	317 (8.9)		374 (8.7)	59 (8.3)		244 (22.4)	123 (16.0)
Hispanic	1177 (1.5)	2567 (4.9)		356 (1.9)	1467 (5.6)		600 (1.4)	817 (4.0)		181 (1.4)	223 (6.3)		30 (0.7)	32 (4.5)		10 (0.9)	28 (3.6)
Asian	2594 (3.3)	652 (1.3)		231 (1.3)	152 (0.6)		1761 (4.1)	396 (1.9)		467 (3.8)	80 (2.2)		121 (2.8)	17 (2.4)		14 (1.3)	7 (0.9)
Native American or Pacific Islander	551 (0.7)	151 (0.3)		109 (0.6)	43 (0.2)		279 (0.7)	67 (0.3)		90 (0.7)	29 (0.8)		44 (1.0)	1 (0.1)		29 (2.7)	11 (1.4)

Results

There were 94,823 cases identified in NSQIP and 59,358 in TOPS meeting our inclusion criteria. Across all 5 procedure categories, TOPS patients were younger (47.9 years versus 50.0 years), more likely to have an ASA class of I or II (92.1% versus 74.6%), more likely to be outpatient (66.0% versus 49.3%), and more likely to be smokers (18.7% versus 11.7%). NSQIP patients were more likely to have diabetes (6.3% versus 4.0%) and more likely to be men (4.8% versus 3.4%). These trends were similar across the 4 procedure groups with few exceptions, such as a slightly older age in TOPS patients undergoing prosthetic breast reconstruction (51.87 years versus 51.12 years) (Table 1).

Table 2 demonstrates the extent of missing data across the 2 datasets and 5 procedure groups. In most categories, the TOPS database had far greater amounts of missing data overall, specifically with regard to age (6.3% versus 0.1%), BMI (40.6% versus 0.8%), ASA class (34.9% versus 0.1%), diabetes status (39.3% versus 0.0%), and smoking status (37.2% versus 0.0%). Within TOPS, free flaps had the greatest percentage of missing data in each of those categories. NSQIP had a higher rate of missing data overall in one category: race (15.6% versus 12.5%). Missing data on race, however, were not higher in NSQIP across all procedures; they were only higher in the breast reduction (28.5 % versus 11.4%) and prosthetic breast reconstruction (10.0% versus 9.8%) cohorts (Table 2).

Table 2.

Summary of Missing Data across Datasets by Surgery Type

	Overall		Breast Reductions		Prosthetic Reconstruction		Autologous Reconstruction		Free Flap		Pressure Ulcer Repair
	NSQIP	TOPS	NSQIP	TOPS	NSQIP	TOPS	NSQIP	TOPS	NSQIP	TOPS	NSQIP	TOPS
Missing age	48 (0.1)	3761 (6.3)	5 (0.0)	2130 (7.2)	4 (0.0)	1169 (5.1)	0 (0.0)	257 (5.4)	23 (0.5)	121 (11.9)	16 (1.3)	84 (7.9)
Missing ASA	99 (0.1)	20,733 (34.9)	26 (0.1)	8990 (30.2)	48 (0.1)	8393 (36.9)	12 (0.1)	2383 (49.8)	9 (0.2)	555 (54.6)	4 (0.3)	412 (38.9)
Missing BMI	717 (0.8)	24,084 (40.6)	238 (0.9)	10,925 (36.7)	311 (0.6)	9584 (42.1)	60 (0.4)	2141 (44.7)	39 (0.8)	698 (68.6)	69 (5.7)	736 (69.5)
Missing diabetes status	0 (0.0)	23,318 (39.3)	0 (0.0)	10,615 (35.7)	0 (0.0)	9038 (39.7)	0 (0.0)	2462 (51.4)	0 (0.0)	652 (64.1)	0 (0.0)	551 (52.0)
Missing admission type	0 (0.0)	84 (0.1)	0 (0.0)	7 (0.0)	0 (0.0)	60 (0.3)	0 (0.0)	17 (0.4)	0 (0.0)	0	0	0
Missing race	14,814 (15.6)	7435 (12.5)	7419 (28.5)	3384 (11.4)	4829 (10.0)	2232 (9.8)	1695 (11.9)	1228 (25.7)	750 (14.7)	303 (29.8)	121 (9.9)	288 (27.2)
Missing smoker status	1 (0.0)	22,068 (37.2)	0 (0.0)	9910 (33.3)	1 (0.0)	8654 (38.0)	0 (0.0)	2410 (50.3)	0 (0.0)	609 (59.9)	0 (0.0)	485 (45.8)
Missing sex	75 (0.1)	0 (0.0)	12 (0.0)	0 (0.0)	50 (0.1)	0 (0.0)	11 (0.1)	0 (0.0)	2 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)

Patients in TOPS with complete data were compared with those with missing data with respect to all surgical complication rates of interest. Overall, patients with complete data had higher rates of dehiscence (5.1% versus 3.5%), surgical infection (2.1% versus 1.7%), medical complication (0.3% versus 0.2%), and partial flap loss (1.2% versus 0.8%) than those with missing data. Where applicable, these trends were similar across all procedures (Table 3).

Table 3.

Comparing Complications in TOPS-EC versus TOPS-WM

	Overall		Breast Reductions		Prosthetic Reconstruction		Autologous Reconstruction		Free Flap		Pressure Ulcer Repair
	TOPS-EC	TOPS-WM	TOPS-EC	TOPS-WM	TOPS-EC	TOPS-WM	TOPS-EC	TOPS-WM	TOPS-EC	TOPS-WM	TOPS-EC	TOPS-WM
N	59,358	29,177	29,746	15,591	22,749	11,315	4787	1853	1017	208	1059	210
Medical complications	110 (0.2)	79 (0.3)	24 (0.1)	17 (0.1)	69 (0.3)	47 (0.4)	9 (0.2)	9 (0.5)	3 (0.3)	2 (1.0)	5 (0.5)	4 (1.9)
Dehiscence	2080 (3.5)	1481 (5.1)	1105 (3.7)	863 (5.5)	551 (2.4)	372 (3.3)	285 (6.0)	194 (10.5)	31 (3.0)	12 (5.8)	108 (10.2)	40 (19.0)
Mortality	18 (0.0)	1 (0.0)	3 (0.0)	1 (0.0)	2 (0.0)	0 (0.0)	4 (0.1)	0 (0.0)	3 (0.3)	0 (0.0)	6 (0.6)	0 (0.0)
Venous thromboembolism	71 (0.1)	34 (0.1)	37 (0.1)	17 (0.1)	19 (0.1)	12 (0.1)	11 (0.2)	5 (0.3)	3 (0.3)	0 (0.0)	1 (0.1)	0 (0.0)
Return to OR	1855 (3.1)	908 (3.1)	419 (1.4)	222 (1.4)	1011 (4.4)	512 (4.5)	311 (6.5)	145 (7.8)	85 (8.4)	23 (11.1)	29 (2.7)	6 (2.9)
Surgical infection	1003 (1.7)	601 (2.1)	344 (1.2)	243 (1.6)	495 (2.2)	281 (2.5)	131 (2.7)	64 (3.5)	23 (2.3)	9 (4.3)	10 (0.9)	4 (1.9)
Total flap loss	155 (0.3)	53 (0.2)	NA	NA	NA	NA	84 (1.8)	33 (1.8)	46 (4.5)	14 (6.7)	2 (0.2)	1 (0.5)
Partial flap loss	490 (0.8)	351 (1.2)	NA	NA	NA	NA	126 (2.6)	87 (4.7)	19 (1.9)	9 (4.3)	13 (1.2)	7 (3.3)

EC, entire cohort; NA, not available; WM, without missing.

When comparing patients with complete data across the 2 databases (unweighted), NSQIP was found to have higher rates of medical complications (2.2% versus 0.3%), mortality (0.1% versus 0.0%), venous thromboembolism (0.5 versus 0.1%), return to OR (6.6% versus 3.1%), and surgical infection (3.0% versus 2.1%). TOPS was found to have higher rates of dehiscence (5.1% versus 1.0%). These trends were similar across the procedures, with the largest percentage differences between NSQIP and TOPS seen in pressure ulcer repair (15.3% versus 1.9% medical complications and 3.5% versus 19.0% dehiscence) (Table 4).

Table 4.

Comparisons of Complications across Datasets for Patients with Complete Demographic/Clinical Data

	Overall			Breast Reductions			Prosthetic Reconstruction			Autologous Reconstruction			Free Flap			Pressure Ulcer Repair
	NSQIP-WM	TOPS-WM	P	NSQIP-WM	TOPS-WM	P	NSQIP-WM	TOPS-WM	P	NSQIP-WM	TOPS-WM	P	NSQIP-WM	TOPS-WM	P	NSQIP-WM	TOPS-WM	P
N	79,399	29,177		18,463	15,591		43,081	11,315		12,541	1853		4285	208		1029	210
Medical complications	1758 (2.2)	79 (0.3)	<0.001	118 (0.6)	17 (0.1)	<0.001	603 (1.4)	47 (0.4)	<0.001	325 (2.6)	9 (0.5)	<0.001	555 (13.0)	2 (1.0)	<0.001	157 (15.3)	4 (1.9)	<0.001
Dehiscence	798 (1.0)	1481 (5.1)	<0.001	126 (0.7)	863 (5.5)	<0.001	261 (0.6)	372 (3.3)	<0.001	194 (1.5)	194 (10.5)	<0.001	181 (4.2)	12 (5.8)	0.369	36 (3.5)	40 (19.0)	<0.001
Mortality	85 (0.1)	1 (0.0)	<0.001	3 (0.0)	1 (0.0)	0.739	5 (0.0)	0 (0.0)	NA	4 (0.0)	0 (0.0)	NA	40 (0.9)	0 (0.0)	NA	33 (3.2)	0 (0.0)	NA
Venous thromboembolism	411 (0.5)	34 (0.1)	<0.001	35 (0.2)	17 (0.1)	0.079	164 (0.4)	12 (0.1)	<0.001	127 (1.0)	5 (0.3)	0.003	77 (1.8)	0 (0.0)	NA	8 (0.8)	0 (0.0)	NA
Return to OR	5225 (6.6)	908 (3.1)	<0.001	397 (2.2)	222 (1.4)	<0.001	2623 (6.1)	512 (4.5)	<0.001	1332 (10.6)	145 (7.8)	<0.001	772 (18.0)	23 (11.1)	0.013	101 (9.8)	6 (2.9)	0.002
Surgical infection	2406 (3.0)	601 (2.1)	<0.001	419 (2.3)	243 (1.6)	<0.001	976 (2.3)	281 (2.5)	0.181	535 (4.3)	64 (3.5)	0.116	413 (9.6)	9 (4.3)	0.015	63 (6.1)	4 (1.9)	0.022

NA, not available; WM, without missing.

When inverse probability of selection weighting was applied to each surgical cohort considered in the TOPS dataset, there was continued discordance in complication rates when comparing the weighted TOPS samples to the NSQIP samples. As will be discussed further, weighting was used to mitigate differences between the datasets with respect to all patient characteristics shared across the datasets. Overall and within each surgery type, rates of medical complication were higher in the NSQIP than in the weighted TOPS, and the differences were statistically significant. In the overall comparison, mortality, venous thromboembolism, return to the OR, and surgical infection rates remained higher in the NSQIP. In certain surgical categories, the rates were higher in the NSQIP but not associated with statistically significant P values. Rates of dehiscence remained higher in the weighted TOPS sample when compared with the NSQIP. Only in the free-flap cohort was the difference in dehiscence not associated with a P value significant at any reasonable level.

DISCUSSION

The use of clinical registries is vital to the quality improvement efforts of our healthcare system by contributing to current and agreed upon care guidelines,[8] evaluating disease-specific outcomes[4,24] and reducing healthcare costs.[25-27] Blind use of data from any given registry is unadvisable. A keen understanding of the patient populations within each registry is vital to making inferences about surgical outcomes.[12,19,28-30] The TOPS and NSQIP databases are 2 robust national databases that can be used for plastic surgery outcomes. Our results demonstrate that the TOPS and NSQIP databases have differing patient populations, which must be understood when using the data for analysis.

As has been highlighted in previous literature, the NSQIP and TOPS exhibit unique strengths and weaknesses.[20,28,29] The validity of the NSQIP has been documented extensively. It has been shown that there is very limited (<2%) disagreement among data collectors and auditors of the NSQIP when considering nearly all of the multiple hundred variables included in the dataset.[3,20,31,32] Moreover, rates of surgical outcomes have been validated in single-institution studies.[31] Importantly, the NSQIP sampling methodology produces what can be considered a random sample of surgical cases conducted at US hospitals, making results interpretable and generalizable to a national population of surgical cases.[21] Though not used nearly as extensively, recent research has validated outcomes in TOPS through cross-validation with CosmetAssure and the NSQIP database across several procedures.[20,33]

Although surgically trained nurses collect surgical cases for the NSQIP, board-certified members of ASPS contribute their own data to TOPS through an electronic data capture. TOPS is beneficial to plastic surgery–specific research because it is not restricted to data collected at hospitals and because many more complications specific to plastic surgery are recorded, such as flap, graft, or prosthesis loss. As such, with many plastic surgery procedures being performed in smaller clinics and outside major hospitals (eg, breast reduction and prosthetic breast reconstruction), it is likely that TOPS more accurately represents the patient populations undergoing such procedures and tracks relevant outcomes.[20,22,34] Furthermore, plastic surgeons may be able to more accurately identify and document postoperative wound-related complications when compared with nurses abstracting data from the medical record. However, because of the nature of data collection in TOPS, there is no guarantee against selective reporting of cases or outcomes. Along with differences in the patient populations between datasets, it is possible that complications are underreported in TOPS. TOPS represents a potentially impactful source for research in plastic surgery, but its cohort and contents need to be well understood to draw generalizable conclusions.

We found considerable differences in the distribution of patient characteristics across NSQIP and TOPS overall and within each surgical category. Much of the discordance in patient characteristics is understandable given our knowledge of the sampling methods in each database. In the NSQIP, patients had higher ASA classification, were older, had a higher BMI, were more often diabetic, and were less often having procedures performed on an outpatient basis. Because cases are recorded at hospitals alone in the NSQIP, it is reasonable that the NSQIP patients appear to have lower health status with respect to available measures. TOPS patients may appear healthier because many of their cases are being performed in clinics and on an outpatient basis. Interestingly, overall and across all surgery types, TOPS patients appear to more often be smokers and white. Broadly speaking, it is evident that the NSQIP and TOPS represent distinct patient populations. Researchers must be cautious about interpreting results and must carefully define the target populations with which they intend to study when using TOPS, in particular.

A noteworthy drawback of the TOPS database is the extent to which patient characteristics are missing or not recorded. For each surgery type, at least one patient characteristic was missing by >33%. The missingness across patient characteristics was as severe as 69.5% of pressure ulcer patients missing BMI information. Meanwhile, only race was missing extensively in the NSQIP, where the issue was most severe among breast reduction patients. For research limited entirely to descriptive purposes, missing data may not be of concern. However, for any analysis intending to ascribe associations or causation, adjustment for demographic and clinical information in statistical methods is paramount to limiting effect estimate bias. If attempting to use TOPS for such research, statisticians should be aware of the extensive missing information. The issue of missing data would be of less concern if key surgical outcomes of interest were of equal prevalence between those with missing data and those with complete clinical and demographic information, and if we knew the patients with complete data did not differ from those of the general population captured by TOPS in terms of health and demographics. However, as is displayed in Table 3, many complication rates differ between the complete and missing data TOPS samples. Since the extent of missing data is too large for available missing data methods (eg, imputation-based methods), researchers must proceed with additional caution about describing cohorts obtained using TOPS subjects with complete information.

The TOPS and the NSQIP databases clearly exhibit distinct patient populations and confront differing degrees of missing data. We sought to better understand whether complication reporting was similar across the 2 datasets and if any differences in reporting could be explained away by mitigating differences between the samples with respect to patient characteristics. For those with complete data, if complication rates could be made comparable in a weighted analysis using the NSQIP as a target population, then it is arguable that such a weighted TOPS could be used for nationally generalizable analyses.

Among complete cases in the raw data, we saw that a larger proportion of patients experienced all complications except mortality and dehiscence in the NSQIP. Mortality was comparable across the 2 datasets, but dehiscence rates were higher in TOPS. Although smokers are more prevalent in TOPS, NSQIP subjects were less healthy with respect to ASA classification score, BMI, and diabetes status. Given that these characteristics are known risk factors for various complications, we would expect complication rates to be higher in the NSQIP.[35,36] However, differences in patient health were mitigated in our weighted analysis. We used inverse probability of selection weights to balance the TOPS population toward the NSQIP population in all variables measured. (See tables, Supplemental Digital Content 1, which displays the balance in covariates achieved between the NSQIP and weighted TOPS for all surgeries, .) Only for race in the free flap patients was the distribution not satisfactorily similar between the weighted TOPS and NSQIP databases. Otherwise we achieved an appreciable level of balance in covariates after weighting for all surgical categories. Table 5 indicates that even when the distribution of all measured patient characteristics in TOPS are weighted to emulate that of the NSQIP, complication rates grew more divergent. As such, it is not clear that the differences in patient populations explain complication rates.

Table 5.

Comparisons of Complications across Datasets for Complete NSQIP and Propensity-weighted Complete TOPS

	Overall			Breast Reductions			Prosthetic Reconstruction			Autologous Reconstruction			Free Flap			Pressure Ulcer Repair
	NSQIP-WM	Weighted TOPS	P	NSQIP-WM	Weighted TOPS	P	NSQIP-WM	Weighted TOPS	P	NSQIP-WM	Weighted TOPS	P	NSQIP-WM	Weighted TOPS	P	NSQIP-WM	Weighted TOPS	P
N	79,399	29,053.7		18,463	15,647.6		43,081	11,396.2		12,541	1833.7		4285	191.9		1029	211
Medical complications	1758 (2.2)	121.4 (0.42)	<0.001	118 (0.6)	22.5 (0.14)	<0.001	603 (1.4)	54.0 (0.47)	<0.001	325 (2.6)	8.1 (0.44)	<0.001	555 (13.0)	1.5 (0.79)	<0.001	157 (15.3)	7.2 (3.43)	0.003
Dehiscence	798 (1.0)	1701.9 (5.86)	<0.001	126 (0.7)	946.3 (6.05)	<0.001	261 (0.6)	416.9 (3.66)	<0.001	194 (1.5)	210.9 (11.50)	<0.001	181 (4.2)	8.5 (4.44)	0.889	36 (3.5)	35.5 (16.84)	<0.001
Mortality	85 (0.1)	1.2 (0.00)	<0.001	3 (0.0)	1.1 (0.01)	0.469	5 (0.0)	0.0 (0.00)	NA	4 (0.0)	0.0 (0.00)	NA	40 (0.9)	0.0 (0.00)	NA	33 (3.2)	0.0 (0.00)	NA
Venous thromboembolism	411 (0.5)	37.4 (0.13)	<0.001	35 (0.2)	17.3 (0.11)	0.080	164 (0.4)	15.8 (0.14)	0.001	127 (1.0)	3.9 (0.21)	<0.001	77 (1.8)	0.0 (0.00)	NA	8 (0.8)	0.0 (0.00)	NA
Return to OR	5225 (6.6)	1121.1 (3.86)	<0.001	397 (2.2)	233.8 (1.49)	<0.001	2623 (6.1)	542.8 (4.76)	<0.001	1332 (10.6)	154.6 (8.43)	0.021	772 (18.0)	24.8 (12.93)	0.198	101 (9.8)	9.8 (4.64)	0.134
Surgical infection	2406 (3.0)	704.8 (2.43)	<0.001	419 (2.3)	271.0 (1.73)	0.002	976 (2.3)	318.3 (2.79)	0.007	535 (4.3)	63.9 (3.48)	0.167	413 (9.6)	11.5 (5.97)	0.271	63 (6.1)	4.9 (2.32)	0.174

Datasets have different strengths, particularly in reporting certain complications (Table 6). The TOPS database could be improved with better surgeon involvement and more complete data input. This could be achieved by incentivizing participation or mandating fields to be completed before case submission. Such efforts should be carefully taken to ensure that participation rates do not decline. Our data can also be used by society leadership to advocate for improvement in the NSQIP or other existing databases to include more plastic surgery–specific outcomes.

Table 6.

Advantages of TOPS and NSQIP Databases

TOPS	NSQIP
Tailored for plastic surgery	Data collected by trained surgical nurses
Captures academic, community, and private practice settings	Auditing to evaluate data reliability
Low financial requirement	Low disagreement rate between reviewers
Better captures plastic surgery–specific variables and outcomes	Captures more demographic data
Allows for additional modules and procedure-specific variables	Has expanded to include 8 additional specialties
Captures BREAST-Q scores	Completeness of data
	Vast number of surgical procedures

CONCLUSIONS

The populations represented by the TOPS and NSQIP databases are significantly different. Such differences may stem from the fact that NSQIP data come from large hospitals, and the TOPS data are largely from private physicians and small groups. TOPS is missing a large amount of patient characteristics data. When assessing only TOPS patients with complete data, most rates of complications increase. Differences in complication rates may also be explained by differences in data collection methodology. After mitigating differences in patient characteristics via a weighted analysis of TOPS, complication rates remained discordant between the 2 sources. Only for dehiscence were complication rates higher in TOPS. The observation that dehiscence rates were higher in TOPS may reflect that surgeons more reliably track wound complications than the nurses contributing to NSQIP.

TOPS can be a key resource for plastic surgical outcomes research. However, careful attention must be paid to the population being studied and how one interprets the missing data problem. If accounting for the variation in patient populations between these datasets, the different strengths that each exhibits in complication reporting can be harnessed to improve plastic surgical outcomes.