Validation of the International Classification of Disease 10th Revision Codes for Kidney Transplant Rejection and Failure.

BACKGROUND: Clinical research requires that diagnostic codes captured from routinely collected health administrative data accurately identify individuals with a disease.
OBJECTIVE: In this study, we validated the International Classification of Disease 10th Revision (ICD-10) definition for kidney transplant rejection (T86.100) and for kidney transplant failure (T86.101).
DESIGN: Retrospective cohort study.
SETTING: A large, regional transplantation center in Ontario, Canada. PATIENTS: All adult kidney transplant recipients from 2002 to 2018. MEASUREMENTS: Chart review was undertaken to identify the first occurrence of biopsy-confirmed rejection and graft loss for all participants. For each observation, we determined the first date a single ICD-10 code T86.100 or T86.101 was recorded as a hospital encounter discharge diagnosis.
METHODS: Using chart review as the gold standard, we determined the sensitivity, specificity, and positive predictive value (PPV) for the ICD-10 codes T86.100 and T86.101.
RESULTS: Our study population comprised of 1,258 kidney transplant recipients. The prevalence of rejection and death-censored graft loss were 15.6 and 9.1%, respectively. For the ICD-10 rejection code (T86.100), sensitivity was 72.9% (95% confidence interval [CI], 66.6-79.2), specificity 97.5% (96.5-98.4), and PPV 83.8% (78.3-89.4). For the ICD-10 graft loss code (T86.101), sensitivity was 21.2% (95% CI, 13.2-29.3), specificity 86.3% (84.3-88.3), and PPV 11.7% (7.0-16.4). LIMITATIONS: Single-center study which may limit generalizability of our findings.
CONCLUSIONS: A single ICD-10 code for kidney transplant rejection (T86.100) was present in 84% of true kidney transplant rejections and is an accurate way of identifying kidney transplant recipients with rejection using administrative health data. The ICD-10 code for graft failure (T86.101) performed poorly and should not be used for administrative health research.

Introduction

Health administrative databases provide a convenient means to study diseases and their outcomes at a population level. Epidemiological research commonly uses diagnostic codes recorded in these databases to identify individuals with diseases.[1] For such research to be valid, one must first appreciate the accuracy of the codes in correctly identifying individuals with the disease of interest.[2,3] It is therefore crucial that diagnostic codes be properly validated using appropriate gold-standard reference measures before studying the corresponding disease on a large scale using administrative databases.[2]

A rejection episode in a kidney transplant recipient is a major adverse event with important long-term implications.[4] Given the overall rarity of kidney transplant recipients in the general population,[5] administrative databases may represent an attractive way to examine kidney transplant rejections within a population. The World Health Organization (WHO) publishes and maintains the International Classification of Diseases (ICD) system, a tool which provides standardized diagnostic classification for diseases and health conditions. New within the Tenth Revision of the WHO ICD (ICD-10) is a code for kidney transplant rejection or failure (T86.1). In Canada, this T86.1 code is broken down into a T86.100 code for kidney transplant rejection and a T86.101 code for kidney transplant failure. The performance of these codes for identifying individuals with kidney transplant rejection or failure has yet to be demonstrated. This would yield valuable information that could allow future studies to examine kidney transplant recipients with adverse graft outcomes on a large scale using health administrative databases.

The purpose of this study is to examine the diagnostic characteristics of the ICD-10 codes for kidney transplant rejection and for kidney transplant failure. We hypothesize a moderate to high sensitivity but a lower specificity for both codes due to false-positive (FP) classification.

Methods

Study Design, Participants, and Setting

This was a retrospective study of all incident adult kidney transplant recipients (≥18 years old at date of transplant) at The Ottawa Hospital (TOH) from January 1, 2002, until December 31, 2018. The date of 2002 was chosen because this is when ICD-10 was adopted in Canada. TOH serves a catchment area of more than 1.2 million individuals in Eastern Ontario and is the only kidney transplant program serving the area. Every recipient (>18 years old) of a kidney transplant performed at TOH was included in the study; there were no exclusions. Each study participant was followed until loss to follow-up (transferred to another program, moved to another province) or end of the study period. The study was approved by TOH Research and Ethics Board. The reporting of this study follows the Standards for the Reporting of Diagnostic accuracy studies (STARD) (Supplemental Table S1).[6]

Data Sources

The study population was identified from the TOH Renal Transplant database.[7] This is a database of all kidney transplants occurring at TOH which is prospectively and monthly updated by trained transplant clerks. It contains the recipient’s medical record number (MRN), the date of transplant, the type of transplant (living or deceased donor), as well as vital information such as date of death, date of graft loss, and date of loss to follow-up. Hospital encounter ICD-10 diagnostic codes were retrieved from the Ottawa Hospital Data Warehouse (OHDW), a data repository of routinely collected health administrative data on patients treated at all campuses of TOH. The OHDW contains clinical and demographic information, including data on admissions, patients, providers, diagnoses, interventions, and laboratory testing, among others, from 1996 onwards. It is a commonly used data source for epidemiological studies conducted at our center.[8-10] The patient-unique MRN was used for linkage between the data sources.

Gold-Standard Definition of Rejection and Failure

The medical chart of every study participant was reviewed by a single reviewer with transplant nephrology training (D.M-.A.) to determine the first occurrence of biopsy-confirmed kidney transplant rejection. The date and type of first rejection episode was entered in an Excel sheet for each study participant. This was done before the knowledge of whether or not the study participant had an ICD-10 diagnostic code for rejection since data on ICD-10 discharge codes was obtained after the completion of chart review. At our institution, routine protocol kidney transplant biopsies are not performed; only patients suspected of having a rejection episode undergo a renal biopsy, all biopsies are ordered by a transplant nephrologist and biopsies may occur as an inpatient or outpatient at the discretion of the ordering nephrologist. The diagnosis of rejection requires histopathological confirmation, and rejections are classified by a renal pathologist based on the Banff criteria. The Banff criteria are a standardized means of nomenclature and classification of kidney transplant pathology which have been adopted world-wide since the 1990s. They are updated every 2 years and are considered the gold-standard diagnostic criteria for kidney transplant rejection throughout the world.[11] Since the study period spans many different Banff criteria, the diagnosis given in the biopsy report would have been made according to Banff criteria in effect at the time of biopsy, and therefore represents the era-appropriate gold standard. Rejection was defined as a histopathological diagnosis of acute cellular rejection [TcMR] (includes acute vascular rejection and borderline acute TcMR rejection) or acute antibody-mediated rejection [AbMR] or transplant glomerulopathy (chronic AbMR). Diagnostic uncertainties regarding rejection diagnosis were reviewed with the senior author G.A.K.

Graft loss (death excluded) was defined as permanent return to dialysis (date of initiation of permanent dialysis, determined retrospectively by a transplant clerk after confirmation of the exact date with a transplant nurse and nephrologist), allograft nephrectomy or re-transplantation. It was ascertained through the TOH transplant database and recorded for each study observation. This was done before the knowledge of whether or not the observation had a code for kidney transplant failure since information on ICD-10 discharge codes was only obtained once graft loss had been ascertained for each study participant. As long as an individual is followed by the TOH transplant clinic, graft loss data are complete, regardless of whether it occurred as an inpatient or outpatient because transplant clerks update vital characteristics (death, graft loss, and loss to follow-up) on a monthly basis.

ICD-10 Codes for Kidney Transplant Rejection and Failure

Our institution employs the Canadian coding standards developed by the Canadian Institute for Health Information (Canadian Coding Standards for ICD-10-CA and CCI to capture diagnoses during hospitalization (available at https://secure.cihi.ca/free_products/CodingStandards_v2018_EN.pdf. Accessed January 15, 2020). We examined the ICD-10 T86.1 code for kidney transplant rejection or failure specified as either T86.100 for kidney transplant rejection or as T86.101 for kidney transplant failure.

For each study participant, we determined the first date of a hospital encounter with a discharge code for kidney transplant rejection (T86.100), and the first date with a discharge code for kidney transplant failure (T86.101). This was only done after completion of chart review and ascertainment of outcomes (rejection and graft loss) for each study participant. The diagnostic code data were retrieved by a data analyst who had no knowledge of the outcomes for the participants. The diagnostic code could be of any type, that is the main (most responsible) diagnosis or a secondary diagnosis to capture any rejection or graft loss event in a given hospitalization. For each study observation receiving a code of interest (T86.100 kidney transplant rejection and T86.101 kidney transplant failure), the medical chart was again reviewed by a single reviewer (D.M-.A.) to confirm that the event of interest actually occurred during the same encounter for which the code was assigned. If the event of interest actually occurred during that encounter, the code was classified as a true positive, whereas if there was no event of interest during that encounter (for example, the actual event occurred at a later hospital encounter), the code was classified as an FP. Also, for any study observation who received a diagnostic code but never had the corresponding event during their follow up, the code was also classified as an FP. For any study individual who had an event of interest but never received a corresponding code, the code was classified as an FN. Finally, for study individuals who never received a diagnostic code for rejection or graft failure and never had an event of interest, the code was classified as a true negative.

Statistical Analysis

Baseline variables at time of transplant surgery for each study observation were determined to provide the characteristics of our population (age, sex, race, cause of end-stage kidney disease [ESKD], number of the current transplant, cumulative panel reactive antibodies [PRAs] pretransplant, presence of comorbidities, type of transplant [living or deceased donor], type of induction therapy received, and type of maintenance immunosuppression used). Means with standard deviations (for parametric continuous variables) and counts with percentages (for categorical variables) were presented.

Using 2 × 2 tables, we calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy ([true positive + true negative]/[whole cohort]) of the diagnostic codes compared to the gold standard of chart review. We calculated the positive likelihood ratio for each code using the formula sensitivity/(1−specificity) as well as the posttest probability.[12] Analyses were done separately for the ICD-10 code T86.100 (kidney transplant rejection) and then for the T86.101 (kidney transplant failure). A post hoc analysis was done for the T86.100 rejection code where the charts of all FP and FN were reviewed to ascertain identifiable reasons for misclassification. We also performed an additional analysis of the rejection code by restricting our study to three different time periods (recipients of transplants occurring from 2002 to 2007 vs 2008 to 2013 vs 2014 to 2018) to explore if there is an era effect on the performance of the T86.100 code. All statistical analyses were conducted using SAS v9.4 (SAS Institute Inc., Cary NC, USA).

Results

Characteristics of the Study Cohort

There were 1,258 kidney transplant recipients with a median follow-up time of 1,405 days (3.8 years) during the study period. The mean age of the transplant recipients was 52 years old, 66% were male, 77% were Caucasian, just under half of the transplants were living donor transplants (45%) and glomerulonephritis was the most frequent cause for ESKD (33%) followed by diabetes (25%; see Table 1). Eighty-seven (6.9%) individuals were lost to follow up at a median of 204 days (interquartile range [IQR], 51-1,509) posttransplant. There were 197 (15.7%) study observations who had a biopsy-confirmed rejection episode, 158 (12.6%) had TcMR and 30 (2.4%) had acute AbMR. There were 114 (9.1%) study observations who had graft loss (death excluded) during their follow-up (Table 2).

Table 1.

Characteristics of the Study Cohort.

Characteristic	Total cohort
No. of kidney transplant recipients	1,258
Age; mean (SD)	52 (14)
Female; no. (%)	454 (36.1)
Living donor transplant; no. (%)	571 (45.4)
Race; no. (%)
Caucasian	967 (76.8)
Black	109 (8.7)
Asian	68 (5.4)
Middle-Eastern	57 (4.5)
Other	57 (4.5)
Cause of end-stage kidney disease; no. (%)
Glomerulonephritis	413 (32.8)
Diabetes	315 (25.0)
Polycystic kidney disease	180 (14.3)
Congenital anomalies of the kidney and urinary tract	101 (8.0)
Other	249 (19.8)
Comorbidity; no. (%)
Diabetes	389 (30.9)
cardiovascular disease (either coronary artery disease, ischemic stroke, congestive heart failure, or atrial fibrillation)	251 (20.0)
Kidney transplant number; no. (%)
1	1,161 (92.3)
2	87 (6.9)
3	8 (0.6)
4	1 (0.1)
5	0 (0)
6	1 (0.1)
Panel reactive antibodies; no. (%)
0%	590 (47)
1%–19%	440 (35)
20%–49%	78 (6)
50%–79%	77 (6)
≥80%	73 (6)
T-cell depleting induction	542 (43.1)
Tacrolimus maintenance	1,043 (82.9)

Table 2.

Rejections and Graft Failures.

	Number of events (%)	Days to occurrence; median (IQR)	1-year cumulative incidence (%)
Rejection	197 (15.7)	117 (8-626)	11.9
t-cell-mediated rejection	158 (12.6)	115.5 (8-492)	10.0
Antibody-mediated rejection	30 (2.4)	14.5 (6-287)	2.0
Graft loss	114 (9.1)	1,108 (262-2,635)	2.7

Note. IQR = interquartile range.

Diagnostic Code Performance

There were 167 (13.3%) study observations who received the T86.100 kidney transplant rejection code during a hospital encounter and 180 (14.3%) who received the T86.101 kidney transplant failure code (Table 3). The measures of diagnostic performance (95% CI) for the T86.100 rejection code were as follows: sensitivity 72.9% (66.6-79.2), specificity 97.5% (96.5-98.4), PPV 83.8% (78.3-89.4), and NPV 95.2% (94.0-96.5). Similar PPVs were found whether the code was received in the first year or after the first year posttransplant (83.2% [76.7-89.8] and 85.7% [75.1-96.3], respectively). The overall accuracy of the T86.100 rejection code was 93.7% and the posttest probability for an actual rejection episode, given a prevalence of rejection of 15.7% in our study cohort, was 84.5%. Similar measures of diagnostic performance were found when restricting to different eras of kidney transplant year (see Supplemental Table S2). For the T86.101 failure code, the measures of diagnostic performance (95% CI) were sensitivity 21.2% (13.2-29.3), specificity 86.3% (84.3-88.3), PPV 11.7% (7.0-16.4), and NPV 92.8% (91.2-94.3). The PPV was 8.0% (3.5-12.6) when the code was received in the first year and 23.3% (10.6-35.9) after the first year posttransplant. The overall accuracy of the T86.101 failure code was 81.2% and the posttest probability for an actual graft failure, given a prevalence of graft loss of 9.1% in our study cohort, was 13.4% (Table 4).

Table 3.

2 × 2 Tables for the International Classification of Disease-10 Kidney Transplant Rejection (T86.100) and Failure (T86.101) Codes.

		Rejection during the corresponding hospital encounter		Total
		Yes	No
T86.100	Yes	140	27	167
	No	52	1,039	1,091
	Total	192	1,066	1,258
		Graft loss during the corresponding hospital encounter		Total
		Yes	No
T86.101	Yes	21	159	180
	No	78	1,000	1,078
	Total	99	1,159	1,258

Table 4.

Performance of the Kidney Transplant Rejection and Failure Codes.

	ICD code present	Prevalence of rejection	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy	PLR	Post-test probability
T86.100 Rejection	167	15.7% (197/1258)	72.9% (66.6 to 79.2)	97.5% (96.5 to 98.4)	83.8% (78.3 to 89.4)	95.2% (94.0 to 96.5)	93.7%	28.8	84.5%
T86.101 Failure	180	9.1% (114/1258)	21.2% (13.2 to 29.3)	86.3% (84.3 to 88.3)	11.7% (7.0 to 16.4)	92.8% (91.2 to 94.3)	81.2%	1.55	13.4%

Note. CI = confidence interval; PPV = positive predictive value; NPV = negative predictive value; PLR = positive likelihood ratio (sensitivity/[1-specificity]).

Reasons for Misclassification of T86.100 Rejection Code

There were 27 instances of FP code assignment; 10 were for acute kidney injury (AKI) for which no graft biopsy was done during the encounter, nine were for situations of suspected rejection which was investigated with a biopsy but for which there was no rejection, five were for instances where the observation had a failing graft and there was suspicion of rejection, but no biopsy was performed, two were assigned for no apparent reason at discharge from the admission for transplant surgery which was without complications, and one was for a case of acute graft dysfunction where rejection was highly suspected but a biopsy could not be done due to hemodynamic instability of the patient. There were 52 instances of FN code assignment; 23 (44%) were for cases of borderline rejection on biopsy, 12 (23%) were for cases of chronic AbMR, and 17 (33%) were for cases of acute TcMR or acute AbMR on biopsy (Table 5).

Table 5.

Reasons for Misclassification of the T86.100 “Rejection” Code.

	No. (%)
False-positive misclassifications (total = 27)
Acute kidney injury for various other causes, no biopsy done	10 (37)
Graft dysfunction investigated by biopsy, but no rejection	9 (33)
Failing graft, suspected rejection (cessation of immunosuppression, nonadherence)	5 (19)
Well-functioning graft in the immediate posttransplant admission	2 (7)
Acute graft dysfunction, suspected rejection but not biopsied	1 (4)
False-negative misclassifications (total = 52)
Borderline rejection on biopsy	23 (44)
Chronic AbMR on biopsy	12 (23)
Acute rejection (t-cell-mediated rejection and/or AbMR) on biopsy	17 (33)

Note. AbMR = antibody-mediated rejection.

Discussion

We determined that the ICD-10 diagnostic code for kidney transplant rejection had a PPV of 84% and an accuracy of 94%, while the ICD-10 diagnostic code for kidney transplant failure had a PPV of 12% and an accuracy of 81%. These results suggest that the code for kidney transplant rejection is a useful tool to identify kidney transplant recipients with rejection using administrative databases. However, the code for kidney transplant failure performs poorly owing to a high number of FP codes and low PPV, making it an inaccurate method for identifying kidney transplant recipients with graft failure.

The T86.100 code for kidney transplant rejection performed well. It had a high accuracy, mostly due to an excellent specificity with few FPs. We identified only 27 cases of a falsely assigned T86.100 rejection code which is less than we would have expected because we initially hypothesized only a moderate specificity for this code. The high accuracy may be attributable to suspected transplant rejection cases being under the care of a nephrologist and/or the requirement for an invasive procedure (biopsy) to occur. This leads to greater likelihood that a hospital coder will properly interpret the documentation in the patient chart and assign the appropriate code. When we examined the reasons for FP classification, we found that in the majority of instances no biopsy had occurred (17/27). In nearly all of these cases, there was some level of graft dysfunction and rejection was on the initial differential diagnosis. In the rest of the FP cases (10/27), a biopsy was done to rule out rejection, but an alternative diagnosis was found (BK nephropathy, calcineurin inhibitor toxicity, acute tubular necrosis, etc.). One way of improving the PPV of the T86.100 code could be to use an algorithm incorporate procedural billing codes, which overall are shown to have high accuracy.[13,14] As the gold-standard for diagnosis of kidney transplant rejection requires a kidney biopsy, one could imagine the use of an algorithm which combines a procedural billing code for kidney biopsy with an ICD-10 T86.100 rejection code. Such an algorithm could eliminate potential FP misclassifications of the common scenario where a transplant patient is admitted with acute graft dysfunction and rejection is initially suspected, but the patient receives treatment (for example, IV fluids) and kidney function improves without ever necessitating a biopsy. On the other hand, such an algorithm would likely come at the expense of more missed cases of rejection (FN).

We found that about 30% of rejection diagnoses were not captured by the ICD-10 T86.100 rejection code. When examining FN classifications, we identified the type of rejection as an important factor. Among the 52 FN T86.100 kidney transplant rejection codes, 23 were cases of borderline rejection and 12 were cases of chronic AbMR. When one of these diagnoses occur, it is conceivable that medical documentation is unclear as to the occurrence of rejection given clinical uncertainty within the transplant community as to whether or not they require specific therapy.[15,16] Also, some clinicians may refer to chronic AbMR as transplant glomerulopathy, which one would not expect a hospital coder to associate with rejection. Consistency and standardization of terminology may improve the FN rate. Overall it appears that the ICD-10 T86.100 code is better suited for capturing cases of more severe, acute rejection episodes.

The ICD-10 T86.101 code for kidney transplant failure performed poorly overall. The sensitivity was low (only 21%), and there were many FP classifications (PPV only 12%). This could be explained by the ambiguity of the term kidney transplant “failure.” Hospital coders may not appreciate the difference between a failing graft with chronic dysfunction or an acutely failing graft due to an acute medical condition which then recovers with treatment, and graft loss requiring the initiation of permanent dialysis. Also, in most instances, the kidney transplant patient with a failing graft who needs to start renal replacement therapy will have this done as an outpatient, without there ever being the chance for a diagnostic code to be assigned. Therefore, the T86.101 is clearly an inappropriate way of identifying kidney transplant patients with graft loss. Thankfully, there are a multitude of high-quality, well-established renal registries which capture kidney transplant patients who develop ESKD requiring initiation of dialysis.[17] In light of our findings, such registries provide a much more accurate way of identifying kidney transplant patients with graft loss.

Our study has a number of strengths and provides the basis for using ICD codes to study kidney transplant recipients with rejection on a large, population level using administrative health data. First, we had a fully inclusive study design which improves the generalizability of our findings. Secondly, the prevalence of rejection in our study population is similar to what is commonly reported in other transplant centers.[18] Since the prevalence of disease may affect the PPV of a diagnostic code for the population being studied, we would not expect much variation in the PPV of the T86.100 rejection code from one center to another in that regard.[19] It is possible that differences in local hospital coding practice could lead to different classifications though. Thirdly, as all kidney biopsies in our institution are ordered by a nephrologist, the reflective chart terminology may be more accurate leading to improved code accuracy. Finally, by systematically reviewing all pathology reports for each study observation, we employed the gold-standard definition of rejection against which to test the performance of the ICD-10 code.

Our study has limitations. Firstly, albeit a large regional institution, it is a single-center study and requires replication to ensure the reliability of our findings. Indeed, there may be important differences in hospital documentation, hospital coder training and coding practices which could lead to different results in terms of code performance. Also, at our center, we do not perform routine biopsies, and we have a higher threshold to biopsy later on after transplant (ex, after the 1-year mark) as can be seen by the fact that the majority of rejections occurred within the first year in our study. Therefore, a center which performs more frequent late biopsies or protocol biopsies may find different results. Second, as the gold-standard definition of rejection was ascertained through chart review by a single reviewer there is the risk for observer bias. We expect this to be negligible though because chart-reviewed rejections are objective diagnoses based on pathological reports, and these were ascertained by a nephrologist with transplant training, before the knowledge of the ICD-10 code classification. Third, some kidney transplant biopsies at our institution are done on an outpatient basis as a day procedure. This means that hospital coders would not have the appropriate documentation to know whether there was rejection or not, which may account for some of the FN results. This limitation could also account for some FP classifications; for example, if an individual had a biopsy done as a day procedure to rule out rejection and the hospital coder assigns a T86.100 rejection code to this encounter, but the biopsy reveals BK nephropathy, it would be an FP classification. Fourth, we have missing data for 87 individuals who were lost to follow up. It is unlikely though that these individuals would have a major impact on our findings for rejection code performance because the majority of rejections occurred before loss to follow-up. Finally, the retrospective nature of our study means that we were unable to inquire with hospital coders the reasons for certain misclassifications. This would yield valuable information that could guide improved documentation in ICD-10 coding manuals and targeted training for hospital coders.

Conclusions

The ICD-10 T86.100 diagnostic code for kidney transplant rejection performs well for identifying kidney transplant patients with rejection of the allograft. The use of administrative databases in kidney transplant research is expanding. Our findings are important because they support the use of ICD-10 codes for conducting epidemiological research on kidney transplant patients having sustained a rejection episode, an adverse event with a potentially major impact on long-term transplant outcomes.

Click here for additional data file.

Supplemental material, sj-pdf-1-cjk-10.1177_2054358120977390 for Validation of the International Classification of Disease 10th Revision Codes for Kidney Transplant Rejection and Failure by David Massicotte-Azarniouch, Manish M. Sood, Dean A. Fergusson and Greg A. Knoll in Canadian Journal of Kidney Health and Disease