Postoperative complications following colectomy for ulcerative colitis: a validation study.

BACKGROUND: Ulcerative colitis (UC) patients failing medical management require colectomy. This study compares risk estimates for predictors of postoperative complication derived from administrative data against that of chart review and evaluates the accuracy of administrative coding for this population.
METHODS: Hospital administrative databases were used to identify adults with UC undergoing colectomy from 1996-2007. Medical charts were reviewed and regression analyses comparing chart versus administrative data were performed to assess the effect of age, emergent operation, and Charlson comorbidities on the occurrence of postoperative complications. Sensitivity, specificity, and positive/negative predictive values of administrative coding for identifying the study population, Charlson comorbidities, and postoperative complications were assessed.
RESULTS: Compared to chart review, administrative data estimated a higher magnitude of effect for emergent admission (OR 2.52 [95% CI: 1.80-3.52] versus 1.49 [1.06-2.09]) and Charlson comorbidities (OR 2.91 [1.86-4.56] versus 1.50 [1.05-2.15]) as predictors of postoperative complications. Administrative data correctly identified UC and colectomy in 85.9% of cases. The administrative database was 37% sensitive in identifying patients with ≥ 1Charlson comorbidity. Restricting analysis to active comorbidities increased the sensitivity to 63%. The sensitivity of identifying patients with at least one postoperative complication was 68%; restricting analysis to more severe complications improved the sensitivity to 84%.
CONCLUSIONS: Administrative data identified the same risk factors for postoperative complications as chart review, but overestimated the magnitude of risk. This discrepancy may be explained by coding inaccuracies that selectively identifying the most serious complications and comorbidities.

Background

Approximately 10% of ulcerative colitis (UC) patients require a colectomy within 10 years of diagnosis [1]. Colectomy for UC is a technically demanding operation associated with morbidity and mortality [2,3]. Patients undergoing elective procedures have lower risk of postoperative mortality, ranging from 0.0% to 1.0% [4-7]. In contrast, mortality in those requiring emergent colectomy was as high as 6.9% [8-10]. Other factors that have been shown to influence postoperative outcomes include older age and comorbidities [11].

Previous studies reporting postoperative outcomes in UC patients have used medical chart review to obtain clinical information. However, these studies are rarely population-based allowing for referral bias and have small sample sizes. Consequently, investigators have relied on administrative databases to study population-based estimates of postoperative outcomes [11]. Administrative databases are time and cost efficient resources but interpretation of results derived from administrative data is dependent on the validity of administrative coding for predictors and outcomes. Therefore, validation of these databases is a priority in health services research [12].

Although numerous studies have used administrative data to study UC outcomes [11,13,14], few have validated the accuracy of administrative data in identifying UC patients who underwent a colectomy. Furthermore, the accuracy of administrative data in characterizing risk factors such as comorbidities is inconsistent. Under-reporting of comorbidities is high[15] and differentiating postoperative complications from pre-admission comorbidities can be challenging. Consequently, inherent misclassification may be present when administrative databases are used to identify preoperative risk factors of postoperative complications [16].

Thus, we compared estimates of the preoperative risk factors (age, emergent colectomy, and comorbidities) associated with postoperative complications in UC patients undergoing colectomy, derived from two data sources: chart review and administrative data. Subsequently, we evaluated the accuracy of administrative databases in defining: 1) the UC study population; 2) preoperative risk factors; and 3) postoperative complications.

Methods

Study Population

The Data Integration, Measurement and Reporting Hospital Discharge Abstract Database (DAD) captures all hospitalizations in the Calgary Health Zone of Alberta Health Services, Canada. The Calgary Health Zone is a population-based health authority under a public, single payer system, with an estimated population of 1.3 million in 2009[17]. The DAD database used the International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) up to March 31, 2001; ICD-10-CA and the Canadian Classification of Health Intervention (CCI) coding have been used since April 1, 2002.

The DAD was searched to identify adult patients (≥18 years) admitted to hospital between January 1, 1996 and December 31, 2007 with a diagnosis of UC (ICD-9 556.X, ICD-10 K51.X). We then identified UC patients who had a code for colectomy (ICD-9-CM 45.7, 45.8 or CCI: 1.NM.87, 1.NM.89, 1.NM.91, 1.NQ.89, 1.NQ. 90). Recognizing that the administrative database may have missed some UC patients who underwent a colectomy, we identified a cohort of patients admitted for an UC flare without a colectomy. All patients with UC at the primary diagnosis coding field and a random subset of patients with UC coded in the second or third diagnostic position were identified. All medical charts of patients identified by the administrative database were reviewed using a standardized, a priori defined electronic data extraction form. Data was extracted by five trained research assistants who were blinded to the original administrative coding. Fifty patient charts were used as a reference standard; all reviewers extracted data on these fifty charts and their abstraction was verified by a gold-standard reference data abstracter (SD) to minimize inter-observer variability.

Outcomes

The primary outcome was occurrence of in-hospital postoperative complications, defined as unexpected medical events occurring between the start of the operation and discharge from hospital. For the chart review, postoperative complications were stratified by severity using the Clavien Classification of Surgical Complications [18] system (See Additional file 1). Each patient was assigned a postoperative status (≥ 1 versus 0 complications) and severity by Clavien class (I-V). For patients experiencing more than one postoperative complication, the most severe complication class was assigned. Complications were also stratified by category: gastrointestinal, cardiovascular, infectious, etc. See Additional file 1: Table S2 for the specific complications comprising each category. In the administrative database, we identified postoperative complications based on pre-defined ICD-9 and CCI codes that have been commonly used to identify postoperative complications [11]. Complication codes used in the analysis can be found in Additional file 1: Table S3.

Variables

Variables extracted from both chart review and administrative data included age at colectomy; emergent versus elective operation; reason for colectomy (UC refractory to medical management, dysplasia or cancer, and acute complication of UC); and pre-admission comorbidities defined by the Charlson-Deyo[19] and Elixhauser [20] indices. Comorbidities were stratified by activity status to identify medical conditions that were managed during the admission. Colectomies were documented as elective if the decision to operate was made prior to hospital admission; in contrast, the decision for emergent colectomy occurred during the admission (e.g. in response to acute life-threatening complications of UC flare or medically refractory disease). In the administrative database, elective colectomies were defined as those coded with an ‘elective’ status, while emergent colectomies were defined using a composite of either ‘emergent’ or ‘urgent’ codes. Age, comorbidity, and admission type were a priori defined as preoperative risk factors and subsequently validated because previous studies have shown that they were associated with postoperative complications in UC patients [11,21].

Data analysis

The administrative coding was validated against the chart review for the study population, admission type, comorbidity status, and postoperative complications. In our primary cohort, we validated the accuracy of administrative data in identifying UC patients undergoing colectomy. Secondarily, a cohort of UC patients admitted for flare without operation was reviewed to detect colectomy patients not captured in our primary cohort. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) with 95% CIs were calculated for UC diagnostic codes alone and the combination of UC + colectomy codes. For patients presenting with UC flare, the analysis was also stratified by UC diagnostic position. In secondary analysis, we validated the accuracy of specific procedural colectomy codes. We also validated the accuracy of administrative data defining emergent or urgent versus elective colectomy. In a sensitivity analysis, we excluded patients admitted urgently, and compared only emergent versus elective patients.

The validity of administrative data in capturing comorbid conditions in colectomy patients was also assessed. We searched the administrative database to identify patients coded with Charlson-Deyo comorbidities and cross-matched these with the true comorbidities identified by chart review. Sensitivity, specificity, PPV, and NPV with 95% CIs were calculated for administrative data predicting whether patients had 0 versus ≥ 1 pre-hospital comorbidities and for each of the 17 specific comorbidities. Subgroup analysis of only comorbidities active on admission was performed. Analyses were repeated for Elixhauser comorbidities (Additional file 1: Table S4-S6.

We assessed the validity of administrative data in capturing in-hospital postoperative complications. Patients coded with complications in the administrative database were cross-matched with those identified by chart review. Sensitivity and specificity with 95% CIs was then calculated under 3 scenarios: 1) none versus any complications; 2) none or Clavien class I versus class II-V complications; and 3) none or class I-II versus class III-V complications. Sensitivity and specificity in capturing specific categories of complications was also determined.

Multivariate logistic regression analysis was performed to examine the association between preoperative risk factors and postoperative complication. In primary analysis, postoperative complication status was defined as none versus any complication. Age (defined as 18–34, 35–64, and ≥ 65 years), comorbidity (0 versus any Charlson comorbidity, and secondarily Elixhauser), and admission type (emergency versus elective) were a priori included into the logistic regression model. Odds ratios with 95% confidence intervals (CI) were calculated for each preoperative risk factor. Two logistic regression models were developed for comparison: 1) data derived from chart review and 2) data derived from the administrative database.

Statistical analyses were performed using SAS statistical software (version 9.2, SAS Institute Inc., Cary, NC). Ethics approval for the study protocol was granted by the Conjoint Health Research Ethics Board at the University of Calgary, study #21833.

Results

Study population

The cohort identification process is illustrated in Figure 1. Search of the administrative database from 1996 to 2007 for patients with UC and colectomy codes found 697 admissions. Excluding unavailable charts (n = 32) and repeated admissions for the same patient (n = 25), we completed chart review on 640 patients and identified 586 patients who underwent colectomy for UC. Among 586 patients, 60% were male and 53% underwent an elective operation. Median age at operation was 40.0 years [interquartile range: 30.0–53.0]. Thirty three percent of UC patients had at least one Charlson comorbidity and 39% experienced an in-hospital postoperative complication following colectomy.

Figure 1

Approach to identifying UC patients who flared and those who underwent colectomy. α “Other Not UC”: Ischemic colitis n = 1, Pseudomembranous colitis n = 3, unspecified n = 12. β “Other Not UC”: Ischemic colitis n = 4, Pseudomembranous colitis n = 7, unspecified n = 23. δ “Previous colectomy”: Record of previous colectomy occurring before the study period 1996–2007 (n = 68), record of previous colectomy occurring in study period (n = 2). * Subset of colectomy patients (n = 15), not captured by administrative definition.

For the non-operative UC flare cohort, 833 admissions were identified as having UC in diagnostic positions 1–3 with the exclusion of colectomy codes. Among these, 569 charts were reviewed; 136 charts were unavailable and128 were repeat admissions. We identified 325 patients who were admitted for a flare of their UC; of these, 298 (91.7%) had UC as the primary diagnosis.

Validation of study population

The administrative database correctly identified both UC and colectomy in 85.9% [83.2–88.5%] of cases (571/665). PPV for specific colectomy codes is found in Table 1. Reasons for misclassification included: repeat admissions (n = 25, 3.8%); patients did not have UC (n = 36, 5.4%); and patients did not undergo colectomy (n = 33, 5.0%). Most commonly, patients were misclassified because the diagnosis was Crohn’s disease (n = 20), ischemic or pseudomembranous colitis (n = 4), or laparoscopic investigative studies rather than colectomy.

Table 1

Validation of Colectomy Procedural Codes

Procedural codes	Definition	Total Codes (n)	Colectomy (n)	PPV(95% CI)
45.7 or 45.8*	Open and other partial excision of large intestine or total intra-abdominal colectomy	294	275	0.94[0.90–0.96]
1.NM.87	Excision partial, large intestine	45	20	0.44[0.31–0.59]
1.NM.89	Excision total, large intestine endoscopic	52	47	0.90[0.79–0.96]
1.NM.91	Excision radical, large intestine	1	1	1.00[0.21–1.00]
1.NQ.89 or 1.NQ.90α	Excision total, rectum or excision total with reconstruction, rectum using open approach with ileum	273	228	0.84[0.79–0.87]
All colectomies		665	571	0.86[0.83–0.88]

* 1 chart identified coded with both 45.7 and 45.8 codes.

α 3 charts identified coded with both 1.NQ.89 and 1.NQ.90 codes.

The administrative database was 94.4% [90.9%–96.6%] sensitive at identifying patients admitted for emergent colectomy; specificity was 76.4% [71.3%–80.8%]. Sensitivity analysis comparing only patients coded as ‘elective’ or ‘emergent’ (i.e. excluded patients admitted under ‘urgent’ status) showed improved specificity (96.3% [93.1%–98.0%]) but decreased sensitivity (85.5% [76.7%–90.7%]).

Searching the non-operative UC flare cohort identified an additional 15 colectomy patients who were missed in the primary search. Overall, the administrative data was 46.6% [42.9%–50.3%] accurate in identifying patients presenting with a flare of UC (325/697). Reasons for misclassification included: prior colectomy (n = 70); patients did not have UC (n = 59), or most commonly, UC was a comorbidity for an unrelated admission (n = 105). The accuracy of administrative data identifying a UC flare varied by diagnostic position: 79.3% [75.2%–83.4%] in the primary diagnostic position; 18.7% [11.3%–26.1%] in the second position; and 8.1% [2.4%–13.9%] in the third position.

Outcome validation

Chart review determined that 39.2% (224/571) of patients undergoing colectomy experienced at least one postoperative complication and 12.3% (70/571) experienced at least one severe postoperative complication (Clavien class III or greater). Sensitivity of administrative data in identifying patients who experienced at least one postoperative complication was 68.3% [61.8%–74.3%] with specificity of 83.6% [79.3%–87.1%], PPV 72.9% [66.5%–78.4%], and NPV 80.3% [75.9%–84.1%]. Results stratified by complication category can be found in Table 2. With advancing complication severity, the sensitivity of the administrative database increased while the specificity decreased (Table 2). The administrative data was the most sensitive at identifying Clavien class III or greater postoperative complications (84.3% [73.6%–91.9%]); however, the specificity was reduced to 69.9% [65.6%–73.9%].

Table 2

Validation of postoperative complication coding in the administrative database. Sensitivity and specificity with 95% CI of administrative data in identifying postoperative complications stratified by complication group and severity (determined using Clavien Classification I-V)

Complication group	Total	Sensitivity	Specificity
Any Complication
0 vs ≥1	224	0.68 [0.62 – 0.74]	0.84 [0.79 – 0.87]
0/Class I vs. Class II-V	154	0.78 [0.71 – 0.84]	0.78 [0.74 – 0.82]
0/Class I-II vs. Class III-V	70	0.84 [0.74 – 0.92]	0.70 [0.66 – 0.74]
Gastrointestinal complications
0 vs ≥1	108	0.56 [0.47 – 0.66]	0.89 [0.85 – 0.91]
0/Class I vs. Class II-V	51	0.67 [0.52 – 0.79]	0.85 [0.81 – 0.88]
0/Class I-II vs. Class III-V	23	0.78 [0.56 – 0.93]	0.82 [0.79 – 0.86]
Infections
0 vs ≥1	78	0.59 [0.47 – 0.70]	0.96 [0.93 – 0.97]
0/Class I vs. Class II-V	69	0.58 [0.45 – 0.70]	0.84 [0.92 – 0.96]
0/Class I-II vs. Class III-V	29	0.66 [0.46 – 0.82]	0.91 [0.88 – 0.93]
Wounds
0 vs ≥1	38	0.50 [0.33 – 0.67]	0.96 [0.94 – 0.98]
0/Class I vs. Class II-V	31	0.48 [0.30 – 0.67]	0.96 [0.94 – 0.97]
0/Class I-II vs. Class III-V	30	0.47 [0.28 – 0.66]	0.96 [0.93 – 0.97]
Renal and Endocrine complications
0 vs ≥1	32	0.66 [0.47 – 0.81]	0.93 [0.91 – 0.95]
0/Class I vs. Class II-V	27	0.74 [0.54 – 0.89]	0.93 [0.91 – 0.95]
0/Class I-II vs. Class III-V	6	1.00 [0.54 – 1.00]	0.91 [0.88 – 0.93]
Cardiovascular disorders
0 vs ≥1	54	0.67 [0.53 – 0.79]	0.97 [0.96 – 0.99]
0/Class I vs. Class II-V	46	0.74 [0.59 – 0.86]	0.97 [0.95 – 0.98]
0/Class I-II vs. Class III-V	18	0.89 [0.65 – 0.99]	0.94 [0.92 – 0.96]
Pulmonary complications
0 vs ≥1	31	0.42 [0.25 – 0.61]	0.97 [0.95 – 0.98]
0/Class I vs. Class II-V	19	0.53 [0.29 – 0.76]	0.96 [0.94 – 0.98]
0/Class I-II vs. Class III-V	14	0.71 [0.42 – 0.92]	0.96 [0.95 – 0.98]
Neurological disorders
0 vs ≥1	12	0.42 [0.15 – 0.72]	0.99 [0.98 – 1.00]
0/Class I vs. Class II-V	6	0.67 [0.22 – 0.96]	0.99 [0.98 – 1.00]
0/Class I-II vs. Class III-V	1	1.00 [0.03 – 1.00]	0.99 [0.97 – 0.99]

Comorbidity validation

Sensitivity of the administrative data in capturing patients with any Charlson comorbidities was 37.3% [30.8%–44.3%] with specificity 97.9% [95.9%–98.9%] (Tables 3 and 4). Sensitivity was improved when the analysis was restricted to active comorbidities alone (62.9% [51.1%–73.2%]); however specificity and PPV were reduced. Results stratified by specific comorbidity are presented in Tables 3 and 4. Overall PPV of the administrative database for identifying the presence of Charlson comorbidities was 90% [82%–95%]. Similar results for Elixhauser comorbidities are presented in Additional file 1: Table S4-S6.

Table 3

Validation of Charlson comorbidity coding in the administrative database. Sensitivity, specificity, PPV and NPV with 95% CI of administrative data in identifying preoperative Charlson comorbidities, stratified by comorbidity type

	n	Sensitivity	Specificity	PPV	NPV
Any Charlson comorbidity*
Totalα	193	0.37 [0.31–0.44]	0.98 [0.96–0.99]	0.90 [0.82–0.95]	0.75 [0.71–0.79]
Active at Admissionβ	70	0.63 [0.51–0.73]	0.93 [0.90–0.95]	0.55 [0.44–0.65	0.95 [0.92– 0.96]
Any malignancy
Total	45	0.40	0.99	0.78	0.95
Active	24	0.75	0.99	0.78	0.99
Cerebrovascular disease
Total	7	0.29	0.99	0.40	0.99
Active	0	N/A	0.99	0.00	1.00
Chronic pulmonary disease
Total	70	0.21	1.00	0.88	0.90
Active	7	0.29	0.97	0.12	0.99
Congestive heart failure
Total	13	0.31	0.99	0.50	0.98
Active	0	N/A	0.99	0.00	1.00
Diabetes w/complication
Total	5	0.60	1.00	1.00	1.00
Active	2	0.50	1.00	0.33	1.00
Diabetes w/o complication
Total	30	0.57	1.00	0.89	0.98
Active	14	0.43	0.98	0.32	0.99
Hemiplegia/paraplegia
Total	2	0.00	1.00	0.00	1.00
Active	1	0.00	1.00	0.00	1.00

α Denotes total number of patients with the specified comorbidity. 193 patients presented with at least 1 Charlson comorbidity. However, sum of patients with each comorbidity type does not equal 193 as patients may have > 1 Charlson comorbidity.

β Comorbidities stratified in chart review by activity on admission.

* No patients identified with AIDS or dementia.

Table 4

Validation of Charlson comorbidity coding in the administrative database continued. Sensitivity, specificity, PPV and NPV with 95% CI of administrative data in identifying preoperative Charlson comorbidities, stratified by comorbidity type

	n	Sensitivity	Specificity	PPV	NPV
Mild liver disease
Totalα	21	0.19	0.99	0.44	0.97
Activeβ	6	0.33	0.99	0.22	0.99
Moderate/severe liver disease
Total	5	0.40	1.00	0.50	0.99
Active	4	0.50	1.00	0.50	1.00
Myocardial infarction
Total	21	0.38	0.99	0.67	0.98
Active	1	1.00	0.98	0.08	1.00
Peptic ulcer disease
Total	11	0.27	0.99	0.50	0.99
Active	3	1.00	0.99	0.50	1.00
Peripheral vascular disease
Total	6	0.17	1.00	0.33	0.99
Active	1	0.00	0.99	0.00	1.00
Renal disease
Total	8	0.13	1.00	0.50	0.99
Active	3	0.33	1.00	0.50	1.00
Rheumatoid arthritis or collagen disease
Total	38	0.13	1.00	0.83	0.94
Active	12	0.17	0.99	0.33	0.98

α Denotes total number of patients with the specified comorbidity. 193 patients presented with at least 1 Charlson comorbidity. However, sum of patients with each comorbidity type does not equal 193 as patients may have > 1 Charlson comorbidity.

β Comorbidities stratified in chart review by activity on admission.

Risk factors for postoperative complications – administrative versus chart review data

Both chart review and administrative data similarly predicted that age ≥ 65 increased the risk of postoperative complications by approximately 2-fold (Table 5). Although emergent admission status was a risk factor for postoperative complication, the odds ratio for emergent admission was higher in the administrative database (OR 2.52 [1.80–3.52]) than that in the chart review (OR 1.49 [1.06–2.09]). The odds ratio for presence of ≥1 Charlson comorbidity was also higher in administrative data (OR 2.91 [1.86–4.56]) as compared to chart data (OR 1.50 [1.05–2.15]) (Table 5).

Table 5

Risk factors for postoperative complications following colectomy in UC patients defined by chart review vs. administrative data

Risk Factor	Chart Data OR (95%CI) n = 586*	Administrative Data OR (95%CI) n = 697*
Age
18–34	1.0	1.0
35–64	0.87 (0.60–1.26)	0.83 (0.58–1.19)
65+	1.97 (1.10–3.52)	2.04 (1.18–3.52)
Admission Type
Elective	1.0	1.0
Emergent	1.49 (1.06–2.09)	2.52 (1.80–3.52)
Charlson Comorbidity
0	1.0	1.0
≥ 1	1.50 (1.05–2.15)	2.91 (1.86–4.56)

OR - odds ratios; CI - confidence interval.

* Cohort of patients identified by administrative data (n = 697) includes all patients coded with UC + colectomy. The cohort used for chart data analysis excludes repeated admissions, patients without UC or colectomy, and unavailable charts (See Figure 1).

Discussion

We conducted this study to evaluate whether outcomes derived from administrative databases accurately represent outcomes obtained from retrospectively reviewing medical charts. Both the administrative database and the chart review identified age, preoperative comorbidities, and emergent surgery as risk factors for postoperative complications following colectomy for UC. However, administrative data overestimated the magnitude of the risk for comorbidities and emergent operations as compared to chart review. Differences in risk estimates were in part explained by misclassification errors associated with the administrative database defining the study population, preoperative risk factors (i.e. comorbidity and emergent colectomy) and postoperative outcome (i.e. complications). The administrative database was more accurate at identifying comorbidities active at admission and the most severe postoperative complications; this selective coding likely biased the risk estimates away from the null hypothesis.

Both clinical [22,23] and administrative database studies [11,24] have identified advancing age, comorbidities, and emergency operations as risk factors for postoperative complications following colectomy and other abdominal surgeries [25-28], though few have evaluated the difference between the two methods[29]. In our analysis both the administrative and chart review data predicted an approximately two-fold increase in complications in those aged ≥ 65 compared to age 18–34 years. Close agreement of OR between administrative and chart data was expected because age is objective and reported with near perfect accuracy in both data sets.

The magnitude of effect for emergent operations was greater with the administrative data as compared to chart review. The administrative database was less specific for identifying emergent colectomy, with a high prevalence of false positives. Sensitivity analysis excluding patients with ‘urgent’ codes demonstrated improved specificity suggesting that the code ‘urgent’ is more aligned with an elective, rather than emergent admission. For example, patients electively admitted for an operation occurring within 24 hours of admission were at times coded as ‘urgent’.

Adaptations of the Charlson and Elixhauser comorbidity indices [20,30] have been validated for risk adjustment of postoperative morbidity and mortality [31-33]. In our analysis, Charlson comorbidities were significantly associated with worse postoperative outcomes; though, the magnitude of effect was greater in the administrative database than chart data. Preferential recording of comorbidities actively managed in-hospital may explain this difference. The sensitivity for most comorbid illnesses was low, but increased when the analysis was restricted to active comorbidities. Our findings were similar to other validation studies that have found poor sensitivity of comorbidity coding[15] and underreporting of chronic comorbidities not requiring treatment [34]. Despite the low sensitivity of administrative data, other studies have found that prediction of in-hospital mortality was identical to indices derived from chart review [31,35]. Additionally, among patients with multiple postoperative complications physicians may record more comorbidities in the discharge summary to explain the poor outcomes, while this may not be detailed in patients with an uncomplicated postoperative recovery.

The administrative database was 86% accurate in identifying patients with UC undergoing colectomy. Additionally, a small subset (n = 15) of UC patients who underwent colectomy were not recorded in the administrative database. Thirumurthi et al. found the sensitivity of the diagnostic code 556 × for hospitalization of UC was 84% [36]. Diagnostic coding for UC may be less accurate than for other conditions; for instance, validations of administrative data in patients presenting with heart failure, acute COPD exacerbations, acute coronary syndromes, and subarachnoid haemorrhage have consistently demonstrated PPV of diagnostic codes exceeding 95% [37-40]. In UC, the lower PPV may reflect uncertainties in diagnosis, especially from Crohn’s disease and other causes of colitis. A previous study also reported higher PPV (96.1%) for colectomy codes[41] compared to our findings, although that validation was performed in a cohort of general surgery patients, with a smaller sample size (n = 56), and included procedural codes for rectal resections (484, 485, 486). In our study, follow-up procedures such as second stage ileopouch anal anastomosis were commonly misclassified as colectomies.

Administrative data did not reliably identify UC patients admitted with a flare without colectomy when the first three diagnostic positions were searched. Although nearly 80% of admissions with UC coded in the primary diagnostic position represented an acute flare of disease, UC recorded in the second or third diagnostic positions represented an acute flare in fewer than 10% of cases. This misclassification error is evident in the literature, as one study demonstrated strengthening of risk estimates when a sensitivity analysis was conducted to exclude Crohn’s disease patients admitted to hospital with a secondary diagnosis of Crohn’s disease [42]. Consequently, prior studies using administrative databases have likely overestimated the true hospitalization rate of UC patients admitted for an acute flare of disease when non-primary diagnostic positions were searched.

The validity of postoperative complications in UC has not been reported. In our study, administrative data was 68% sensitive in identifying patients experiencing at least one complication after colectomy. Previous studies have also shown underreporting of complications in administrative data [43-47]. Misclassification of postoperative complications contributed to the discrepancy observed between administrative and chart review data. The accuracy of administrative data in coding postoperative complications was correlated to complication severity: sensitivity increased when less severe complications were excluded from the analysis while the specificity decreased. Administrative data poorly identified minor complications (i.e. Clavien I), but captured the more severe and clinically significant postoperative complications. These findings were similar to our comorbidity validation, supporting the notion that administrative databases miss comorbidities and complications that likely have less clinical impact.

Misclassification of post-operative complications was predominantly due to the challenge in differentiating a postoperative complication from a comorbidity or a preoperative in-hospital complication. For example, UC patients who underwent colectomy and were coded for pulmonary embolism were recorded as a false positive if the pulmonary embolism was diagnosed before the colectomy was performed.

Several limitations of our study should be considered. First, the chart review was retrospective and not all clinical information may have been documented in the charts. As we comprehensively reviewed only the current admission, other comorbidities may have been missed. Second, we only had access to administrative codes for the patient’s hospitalization for colectomy; searching prior admissions may have improved the sensitivity of administrative coding, particularly for comorbidities. This provides an area for future study that may be explored in other datasets. Third, variation between reviewers was unavoidable although we attempted to limit inter-observer variability. Fourth, a small portion (2.6%) of UC patients coded for a flare but not colectomy actually underwent colectomy when the chart was reviewed. Conceivably, UC patients who underwent colectomy may not have been coded for either UC or colectomy, though this misclassification error is likely far less than 2.6%. Fifth, our sample size was sufficient to evaluate the overall validity of administrative data, but uncommon comorbidities and complications could not be validated. Similarly, large administrative database studies have the power to stratify comorbidity as a categorical variable (i.e. 0, 1, 2, or ≥3 comorbidities), but the prevalence of multiple comorbidities in our cohort was too low to accurately perform this subgroup analysis. Finally, the administrative database reflects the quality associated with Calgary’s DAD and thus, may not be generalized to other hospitalization databases. However, Calgary’s DAD is comprehensive, has been widely used and validated for health service research[41], and has demonstrated generalizability in different settings. For example, a recent study demonstrated that Charlson comorbidities predicted in-hospital mortality similarly in Calgary’s hospital DAD as compared to hospitalization databases in France, New Zealand, Japan, Switzerland, and Australia [48]. Thus, the data from this study should reflect practices and outcomes of other administrative databases and at minimum should motivate others to test the validity of local administrative databases.

Conclusions

Administrative data identified the same risk factors (advanced age, emergency admission, and comorbidities) for postoperative complications as chart review. However, the risk estimates were biased away from the null by the administrative database. The discrepancy in risk estimates may be explained by inaccuracies in defining the study population, complications, and comorbidities. Administrative data more accurately identified severe postoperative complications and comorbidities actively managed during the admission. Thus, despite the imperfect validity of administrative data, identified comorbidities and complications were likely the most clinically meaningful. Administrative databases are valid tools for IBD research, but the general inferences drawn from risk estimates should be interpreted in the context of limitations associated in identifying the study population, risk factors, and postoperative complications.

Competing interests

The authors declare that they have no competing interests.

Author contributions

All authors have reviewed and approved this manuscript. Study design and planning, data interpretation, manuscript drafting and approval: Dr. GK. Study planning, data collection and interpretation, manuscript drafting and approval: CMa. Study planning, data collection, manuscript approval: MC, M-CP, and MP. Data analysis and manuscript approval: JH. Study planning, data collection and interpretation, manuscript approval: Dr. SDS. Manuscript drafting and approval: Dr. RP, Dr. SG, Dr. RM, and Dr. HQ.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-230X/12/39/prepub

Additional file 1

Tables S1-S6.For Table S1, please see reference citation number [18], Dindo 2004.

Click here for file