Validity of the International Classification of Diseases, Tenth Revision code for acute kidney injury in elderly patients at presentation to the emergency department and at hospital admission.

OBJECTIVE: To evaluate the validity of the International Classification of Diseases, Tenth Revision (ICD-10) code N17x for acute kidney injury (AKI) in elderly patients in two settings: at presentation to the emergency department and at hospital admission.
DESIGN: A population-based retrospective validation study.
SETTING: Southwestern Ontario, Canada, from 2003 to 2010. PARTICIPANTS: Elderly patients with serum creatinine measurements at presentation to the emergency department (n=36 049) or hospital admission (n=38 566). The baseline serum creatinine measurement was a median of 102 and 39 days prior to presentation to the emergency department and hospital admission, respectively. MAIN OUTCOME MEASURES: Sensitivity, specificity and positive and negative predictive values of ICD-10 diagnostic coding algorithms for AKI using a reference standard based on changes in serum creatinine from the baseline value. Median changes in serum creatinine of patients who were code positive and code negative for AKI.
RESULTS: The sensitivity of the best-performing coding algorithm for AKI (defined as a ≥2-fold increase in serum creatinine concentration) was 37.4% (95% CI 32.1% to 43.1%) at presentation to the emergency department and 61.6% (95% CI 57.5% to 65.5%) at hospital admission. The specificity was greater than 95% in both settings. In patients who were code positive for AKI, the median (IQR) increase in serum creatinine from the baseline was 133 (62 to 288) µmol/l at presentation to the emergency department and 98 (43 to 200) µmol/l at hospital admission. In those who were code negative, the increase in serum creatinine was 2 (-8 to 14) and 6 (-4 to 20) µmol/l, respectively.
CONCLUSIONS: The presence or absence of ICD-10 code N17× differentiates two groups of patients with distinct changes in serum creatinine at the time of a hospital encounter. However, the code underestimates the true incidence of AKI due to a limited sensitivity.

Validation of administrative database codes is a prerequisite to their optimal use in research.

The aim of this study was to describe the validity of the International Classification of Diseases, Tenth Revision (ICD-10) code N17× for acute kidney injury (AKI) compared with a reference standard based on changes in serum creatinine.

The ICD-10 code N17× for AKI has a moderate sensitivity and high specificity.

The sensitivity of the N17× code improves for more severe forms of AKI.

The code was successful in identifying a group of patients admitted to hospital with a median increase in serum creatinine of 98 µmol/l.

This is the first study to provide information on the diagnostic performance of ICD-10 code N17× for AKI using laboratory values as the reference standard.

It was a large population-based validation study that included serum creatinine measurements from 12 hospitals.

Future validation studies in younger patients are required.

Background

Healthcare administrative databases can provide researchers and policy makers with information on a large number of patients in an efficient manner. When using these data resources for clinical or health services research, the validity of the research depends upon the accuracy of the diagnostic and procedural codes that have been recorded.1 However, the accuracy of coding is not guaranteed because administrative databases are not primarily intended for research.2 Consequently, understanding the validity of administrative codes is a prerequisite to their optimal use in the assessment of patient outcomes.

Clinically, acute kidney injury (AKI) is characterised by an abrupt decline in the renal function that may result in disordered fluid, acid–base and electrolyte homeostasis and retention of waste products from nitrogen metabolism, such as creatinine and urea and/or a decreased urine output.3–5 Two systems for defining and quantifying the severity of AKI are widely used: the Acute Kidney Injury Network (AKIN) classification6 and the Risk-Injury-Failure-Loss-ESRD (RIFLE) criteria.7 These staging systems define AKI severity according to absolute and relative (percentage) increases in serum creatinine, a blood test universally used for indicating kidney function. While the incidence of AKI is dependent on the definition used, it is recognised that this condition is common, affecting 2–9% of patients at hospital admission.8–11 Moreover, patients who develop AKI have both poor short-term and long-term outcomes and their care is expensive.8 9 12–20

The purpose of the present study was to evaluate the accuracy of the International Classification of Diseases, Tenth Revision (ICD-10) code N17× for AKI for applications in clinical and health services research, particularly in pharmacoepidemiological studies. We compared this code against changes in serum creatinine concentration in two settings: (1) at presentation to the emergency department and (2) at hospital admission. In addition, we investigated the effect of baseline chronic kidney disease (CKD) status on the diagnostic performance of the code in the two settings. Based on the findings of a previous validation study on ICD-9 codes, we anticipated the sensitivity for ICD-10 code N17× would be low, improving with more severe definitions of AKI.8 10 21 22 Moreover, we expected higher sensitivity in patients with CKD than those without, as the former typically have larger absolute increases in serum creatinine for a given amount of AKI.

Methods

Study design and setting

We conducted a population-based retrospective validation study using Ontario's linked healthcare administrative databases and laboratory data from Southwestern Ontario. All residents receive universal access to hospital and physician services under a single provincial payer system, providing a comprehensive set of health administrative data.

Using a diagnostic test assessment framework, we obtained diagnostic performance characteristics (sensitivity, specificity, positive-predictive value and negative-predictive value) of various diagnostic coding algorithms for ICD-10 code N17×, which is defined as ‘acute renal failure’. We used changes in serum creatinine from the baseline value as the reference standard (see online supplementary table S1 for a sample two-by-two table). Moreover, we compared the change in serum creatinine between patients who were N17× code positive with those who were N17× code negative.

Our protocol was approved by the institutional review board at Sunnybrook Health Sciences Centre (Toronto, Ontario). The relevant datasets are held at the Institute for Clinical Evaluative Sciences. The reporting of this study follows guidelines set out for studies assessing diagnostic accuracy (see online supplementary table S2).23

Data sources

Patient records from the seven databases were linked using encrypted unique identifiers. We identified laboratory measurements, including serum creatinine, using a system that keeps patients’ electronic medical records (Cerner, Kansas City, Missouri, USA).24 This system contains inpatient, outpatient and the emergency department laboratory measurements for 12 Southwestern Ontario hospitals. For a subpopulation, we also obtained previous laboratory measurements from Gamma Dynacare, a provider of outpatient laboratory services to residents in Southwestern Ontario. We obtained inpatient and emergency department patient diagnostic information from the Canadian Institute of Health Information (CIHI) Discharge Abstract Database (DAD) and National Ambulatory Care Reporting System database (NACRS), respectively. We obtained information on inpatient and outpatient physician services from the Ontario Health Insurance Plan database (OHIP). We ascertained patients’ demographic information from the Ontario's Registered Persons Database (RPDB) and prescribed drug use for patients 65 years of age and older from the Ontario Drug Benefit database (ODB). These databases have been used extensively to research health outcomes and health services.25–28

Accrual of elderly patients in Two settings: at presentation to the emergency department and at hospital admission

We developed two separate cohorts. The first cohort consisted of patients with a serum creatinine measurement at presentation to the emergency department (Emergency Department Cohort). The second cohort consisted of patients with a serum creatinine measurement at hospital admission (Hospitalised Cohort). All patients had a baseline serum creatinine measurement (described below). The period of accrual was from 1 June 2003 to 30 September 2010.

We restricted cohort entry to patients 66 years of age and older to ensure at least 1 year of baseline prescription records for all patients. We excluded the following patients from both cohorts: (1) those who had end-stage renal disease (defined by the receipt of dialysis in the 120 days prior to the hospital encounter), (2) those who received a kidney or liver transplant in the 5 years prior to the hospital encounter, (3) those whose date of serum creatinine measurement (from Cerner) and the hospital encounter (from CIHI-DAD) did not align (less than 1.4% patients excluded for this reason; see online supplementary figure S1 for explanation) and (4) those without at least one serum creatinine measurement in the 7–365 days prior to the hospital encounter to serve as the baseline value. In cases where a patient had multiple baseline measurements we selected the most recent one. Additional selection criteria for each cohort are described below and illustrated in online supplementary figure S1.

We excluded patients who did not have a serum creatinine measurement in the emergency department from entry to the Emergency Department Cohort. We excluded the following patients from entry to the Hospitalised Cohort: (1) those with a hospital admission that resulted in a stay greater than 90 days (to ensure we had data for the full hospital admission to the time of discharge, that is, particularly for patients accrued in the second half of the year 2010) and (2) those without a serum creatinine measurement either in the emergency department prior to hospital admission or during the first 2 days of hospital admission. When multiple eligible hospital presentations were identified for a given patient over the study period, we randomly selected one hospital encounter for each cohort.

Within the two cohorts, we identified patients diagnosed with CKD (assessed by the presence of ICD-10 code N18x defined as ‘chronic kidney disease’) in the 5 years prior to their hospital encounter to evaluate potential differential classification of AKI in patients with baseline CKD compared with those without.

The reference standard: serum creatinine-based definitions of AKI

Measuring changes in serum creatinine is the most common method of identifying AKI in clinical practice. We adapted the reference standard used in this study from four widely used serum creatinine-based definitions of AKI: (1) AKIN Stage 1 or greater: ≥27 µmol/l (0.3 mg/dl) or 50% increase in serum creatinine concentration from the baseline, (2) RIFLE Risk: ≥1.5-fold increase in serum creatinine concentration from the baseline, (3) RIFLE Injury: ≥2-fold increase in serum creatinine concentration from the baseline and (4) RIFLE Failure: ≥3-fold increase in serum creatinine concentration from the baseline or a baseline serum creatinine concentration ≥354 µmol/l (4.0 mg/dl) with ≥44 µmol/l (0.5 mg/dl) increase from the baseline.6 7 A Roche Modular Ion Selective Electrode system (Basel, Switzerland) was used to measure serum creatinine.

For the Emergency Department Cohort, we categorised the difference between patients’ peak (highest) serum creatinine concentration at presentation to the emergency department and the baseline serum creatinine concentration into the serum creatinine-based definitions. To be classified as having AKI in the Hospitalised Cohort, patients had to have a ≥27 µmol/l (0.3 mg/dl) or 50% increase in serum creatinine concentration from their baseline value at the emergency department or during the first 2 days of hospital admission. This was done to ensure all patients classified as having AKI by the serum creatinine-based definitions manifested AKI at hospital admission rather than developing the condition de novo during the hospital stay. However, there may be a delay from the time of injury to when the peak serum creatinine concentration is realised. Thus, in those with AKI, we categorised the severity as defined by the serum creatinine-based definitions, using the peak serum creatinine concentration in either the emergency department or in the first 5 days of hospital admission.

ICD-10 coding administrative database algorithms for AKI

Following a discharge from the hospital, trained coders review all charts to record appropriate diagnosis codes and their associated attributes. The coders follow the Canadian Coding Standards developed by CIHI.29 According to CIHI's guidelines, the coders are not permitted to interpret laboratory measurements, but can record a condition based on laboratory findings if the physician documents the condition as diagnosed in the patient's chart. For ambulatory care records (included in CIHI-NACRS), coders are allowed to include up to 10 diagnoses per visit. The first diagnosis listed is the main problem for the patient's visit that required evaluation and/or treatment or management as determined by the physician at the end of the visit. For hospitalisation records (included in CIHI-DAD), coders may record up to 25 conditions using ICD-10 diagnostic codes. Additionally, they must indicate diagnosis type ‘M’ for the condition that was most responsible for the greatest portion of the length of stay or used the greatest amount of resources. They may also indicate diagnosis type ‘1’ for any condition that existed prior to the admission and was treated during the hospital stay.29

In this study, we tested two unique algorithms to identify patients with AKI at presentation to the emergency department and three unique algorithms to identify patients at hospital admission. Each of these algorithms used the ICD-10 code N17×, but varied the possible diagnosis types. In the Emergency Department Cohort, we examined the code: (1) as the main problem (referred to as ‘main diagnosis’) or (2) in any of the 10 potential diagnostic fields with any diagnosis type (referred to as ‘all diagnoses’). In the Hospitalised Cohort, we examined the code: (1) as the diagnosis type of ‘M’ (most responsible; referred to as the ‘most responsible diagnosis’), (2) as the diagnosis type of ‘1’ (pre-admit comorbidity; referred to as ‘admission diagnosis’) or (3) in any one of the 25 potential diagnosis fields with any diagnosis type (referred to as ‘all diagnoses’).

Statistical analysis

We used descriptive statistics to summarise demographic characteristics, comorbidities, prescription drug claim information and baseline laboratory measurements for patients in both settings. We calculated sensitivity, specificity, positive-predictive value and negative-predictive value for each diagnostic coding algorithm (formulas presented in online supplementary table S1). We calculated 95% CI for single proportions using the Wilson Score method.30 We performed these calculations against the four reference standard definitions. We expressed the changes in patient serum creatinine concentration from the baseline as medians with IQR and we compared the means using the Mann-Whitney test. We conducted all analyses using the SAS (Statistical Analysis Software) V.9.2 (SAS Institute Incorporated, Cary, North Carolina, USA, 2008).

Results

We identified a total of 36 049 patients for the Emergency Department Cohort and 38 566 patients for the Hospitalised Cohort. Baseline characteristics of the two cohorts are presented in table 1, including the proportion of patients who satisfied different definitions of AKI. For example, 294 (0.8%) and 567 (1.5%) patients satisfied the RIFLE Injury definition of AKI (defined by ≥2-fold increase in serum creatinine concentration), in each cohort, respectively.

Table 1

Baseline characteristics for patients in the Emergency Department and Hospitalised Cohorts

	Emergency Department Cohort (n=36049)	Hospitalised Cohort (n=38566)
Demographics
Median age (IQR), years	77 (72–83)	76 (71–82)
Women (n (%))	19262 (53.4)	19070 (49.4)
Income Quintile(n (%))
One (lowest)	7678 (21.3)	8027 (20.8)
Two	7306 (20.3)	7765 (20.1)
Three (middle)	7062 (19.6)	7654 (19.8)
Four	6110 (16.9)	6797 (17.6)
Five (highest)	7301 (20.3)	7816 (20.3)
Year of Cohort Entry(n (%))
2003–2004	3648 (10.1)	6733 (17.5)
2005–2006	8348 (23.2)	9256 (24.0)
2007–2008	11954 (33.2)	11380 (29.5)
2009–2010	12099 (33.6)	11197 (29.0)
Rural location(n (%))	5397 (15.0)	7165 (18.6)
Resident in a long-term care facility(n (%))	1454 (4.0)	1298 (3.4)
Comorbidities* (n (%))
Chronic kidney disease†	1526 (4.2)	1632 (4.2)
Diabetes mellitus‡	8497 (23.6)	8650 (22.4)
Peripheral vascular disease	1137 (3.2)	2077 (5.4)
Coronary artery disease§	16847 (46.7)	18844 (48.9)
Congestive heart failure	8860 (24.6)	9224 (23.9)
Stroke/transient ischaemic attack	1434 (4.0)	1467 (3.8)
Chronic liver disease	837 (2.3)	1074 (2.8)
Medication Use* (n (%))
Angiotensin-converting enzyme inhibitor	13781 (38.2)	14859 (38.5)
Angiotensin-receptor blocker	6540 (18.1)	6514 (16.9)
Potassium sparing diuretic	3643 (10.1)	3949 (10.2)
Non-potassium sparing diuretic	16308 (45.2)	17145 (44.5)
Calcium channel blocker	11785 (32.7)	12553 (32.5)
β-Adrenergic antagonist	13646 (37.9)	14662 (38.0)
Statins	15706 (43.6)	16602 (43.0)
NSAIDs (excluding aspirin)	6520 (18.1)	7761 (20.1)
Anticonvulsants	2297 (6.4)	2244 (5.8)
Antidepressants	9187 (25.5)	8938 (23.2)
Antipsychotics	1883 (5.2)	1692 (4.4)
Benzodiazepines	9035 (25.1)	9414 (24.4)
Antineoplastics	2217 (6.1)	2377 (6.2)
Thyroid hormone	6172 (17.1)	6150 (15.9)
Baseline laboratory measurements¶
Serum creatinine concentration, µmol/L, median (IQR)	91 (75–113)	90 (75–114)
eGFR ml/min/1.73 m2,** median (IQR)	61 (46–75)	62 (47–77)
eGFR category(n (%))
≥60 ml/min/1.73 m2	18382 (51.0)	20716 (53.7)
45–59 ml/min/1.73 m2	9043 (25.1)	9011 (23.4)
30–44 ml/min/1.73m2	5622 (15.6)	5633 (14.6)
15–29 ml/min/1.73m2	2415 (6.7)	2537 (6.6)
<15 ml/min/1.73m2	587 (1.6)	669 (1.7)
Urine dipstick protein(n (%))
negative	4186 (84.0)	3252 (81.4)
0.3g/l	415 (8.3)	409 (10.2)
1.0g/l	296 (5.9)	257 (6.4)
≥3.0g/l	87 (1.7)	79 (2.0)
Serum sodium concentration, mmol/l, median (IQR)	139 (137–142)	139 (137–141)
Serum potassium concentration, mmol/l, median (IQR)	4.0 (4.0–5.0)	4.0 (4.0–5.0)
AKI definitions for all patients(n (%))
AKIN Stage 1 or greater	5312 (14.7)	6879 (17.8)
RIFLE Risk	473 (1.3)	884 (2.3)
RIFLE Injury	294 (0.8)	567 (1.5)
RIFLE Failure	527 (1.5)	920 (2.4)
AKI definitions for patients with CKD† (n (%))
AKIN Stage 1 or greater	524 (34.3)	644 (39.5)
RIFLE Risk	25 (1.6)	65 (4.0)
RIFLE Injury	12 (0.8)	41 (2.5)
RIFLE Failure	154 (10.1)	246 (15.1)

*Comorbidities and medication usage in the 5 and 6 months preceding the hospital encounter were considered, respectively.

†CKD was assessed by the ICD-10 code N18×, defined as ‘chronic kidney disease’.

‡Diabetes mellitus was assessed by the diabetic medication use in the previous 6 months.

§Coronary artery disease includes the receipt of coronary artery bypass graft surgery, percutaneous coronary intervention and diagnoses of angina.

¶The baseline measurements for serum creatinine were taken at a median (IQR) of 102 (41–204) and 39 (16–128) days prior to the hospital encounter for the Emergency Department Cohort and the Hospitalised Cohort, respectively. Baseline urine protein and serum sodium and potassium were available for a subset of patients. Emergency Department cohort: A total of 4984, 29 746 and 30 040 patients had a baseline urine protein and serum sodium and potassium measurement available in the 7 to 365 days prior to the index date, respectively. Hospitalised cohort: A total of 3997, 34 407 and 34 538 patients had a baseline urine protein and serum sodium and potassium measurements available in the 7–365 days prior to the index date, respectively

**eGFR was calculated using the CKD-Epi equation.

CKD-Epi equation:141×min((serum creatinine in μmol/L/88.4)/κ, 1)α×max((serum creatinine in μmol/L/88.4)/κ, 1)−1.209×0.993Age×1.018 (if female)×1.159 (if an African–American) κ=0.7 for females and 0.9 for males, α=−0.329 for females and −0.411 for males, min=the minimum of Scr/κ or 1, max=the maximum of Scr/κ or 1. Racial information was not available in our data sources and all patients were assumed not to be of a non African–Canadian race. This was a reasonable assumption; as of 2006, African–Canadians represented less than 7% of the Ontario population. Source: http://www12.statcan.ca/census-recensement/2006/dp-pd/hlt/97-562/index.cfm?Lang=E

eGFR, estimated glomerular filtration rate.

The diagnostic performance of the various coding algorithms is presented in table 2. For both types of hospital encounters, ‘all diagnoses’ was the best performing ICD-10 N17× coding algorithm. At presentation to the emergency department, the sensitivity of the ICD-10 code for the RIFLE Injury definition of AKI (≥2-fold increase in serum creatinine concentration from the baseline) was 37.4% (95% CI 32.1% to 43.1%).

Table 2

Diagnostic performance characteristics of three different algorithms for ICD-10 code N17× using four different serum creatinine-based definitions of AKI as the reference standard

		Diagnostic performance characteristics (95% CI)
Diagnostic coding algorithm	Definition	Emergency Department Cohort	Hospitalised Cohort
All diagnoses	AKIN Stage 1	Sn=7.2 (6.6 to 8.0)	Sn=21.8 (20.9 to 22.8)
		Sp=99.9 (99.8 to 99.9)	Sp=98.4 (98.2 to 98.5)
		PPV=90.4 (87.2 to 92.8)	PPV=74.2 (72.3 to 76.1)
		NPV=86.2 (85.8 to 86.5)	NPV=85.3 (84.9 to 85.6)
	RIFLE Risk	Sn=30.4 (26.5 to 34.7)	Sn=56.4 (53.2 to 59.7)
		Sp=99.2 (99.1 to 99.3)	Sp=96.0 (95.8 to 96.2)
		PPV=33.8 (29.5 to 38.4)	PPV=24.7 (22.8 to 26.6)
		NPV=99.1 (99.0 to 99.2)	NPV=98.9 (98.8 to 99.1)
	RIFLE Injury	Sn=37.4 (32.1 to 43.1)	Sn=61.6 (57.5 to 65.5)
		Sp=99.1 (99.0 to 99.2)	Sp=95.6 (95.4 to 95.8)
		PPV=25.8 (21.9 to 30.2)	PPV=17.3 (15.7 to 19.0)
		NPV=99.5 (99.4 to 99.6)	NPV=99.4 (99.3 to 99.5)
	RIFLE Failure	Sn=30.2 (26.4 to 34.2)	Sn=59.1 (55.9 to 62.3)
		Sp=99.2 (99.2 to 99.3)	Sp=96.1 (95.9 to 96.3)
		PPV=37.3 (32.9 to 42.0)	PPV=26.9 (25.0 to 28.9)
		NPV=99.0 (98.9 to 99.1)	NPV=99.0 (98.9 to 99.1)
Main diagnosis/most responsible diagnosis	AKIN Stage 1	Sn=4.1 (3.6 to 4.6)	Sn=5.1 (4.6 to 5.7)
		Sp=100.0 (99.9 to 100.0)	Sp=99.9 (99.8 to 99.9)
		PPV=94.7 (91.0 to 97.0)	PPV=90.7 (87.4 to 93.2)
		NPV=85.8 (85.4 to 86.1)	NPV=82.9 (82.5 to 83.3)
	RIFLE Risk	Sn=21.4 (17.9 to 25.3)	Sn=18.7 (16.2 to 21.4)
		Sp=99.6 (99.6 to 99.7)	Sp=99.4 (99.3 to 99.5)
		PPV=44.5 (38.2 to 51.0)	PPV=42.5 (33.7 to 47.5)
		NPV=99.0 (98.9 to 99.1)	NPV=98.1 (98.0 to 98.3)
	RIFLE Injury	Sn=27.9 (23.1 to 33.3)	Sn=22.8 (19.5 to 26.4)
		Sp=99.6 (99.6 to 99.7)	Sp=99.3 (99.2 to 99.4)
		PPV=36.1 (30.2 to 42.6)	PPV=33.3 (28.7 to 38.1)
		NPV=99.4 (99.3 to 99.5)	NPV=98.9 (98.7 to 99.0)
	RIFLE Failure	Sn=22.0 (18.7 to 25.7)	Sn=22.9 (20.3 to 25.8)
		Sp=99.6 (99.5 to 99.7)	Sp=99.5 (99.5 to 99.6)
		PPV=51.1 (44.6 to 57.5)	PPV=54.4 (49.4 to 59.3)
		NPV=98.9 (98.7 to 99.0)	NPV=98.1 (98.0 to 98.3)
Admission diagnosis	AKIN Stage 1	n/a	Sn=15.8 (15.0 to 16.7)
			Sp=99.2 (99.1 to 99.3)
			PPV=81.6 (79.4 to 83.6)
			NPV=84.5 (84.1 to 84.8)
	RIFLE Risk		Sn=43.1 (39.9 to 46.4)
			Sp=97.5 (97.3 to 97.6)
			PPV=28.5 (26.2 to 31.0)
			NPV=98.6 (98.5 to 98.8)
	RIFLE Injury		Sn=48.3 (44.2 to 52.4)
			Sp=97.2 (97.0 to 97.4)
			PPV=20.5 (18.4 to 22.8)
			NPV=99.2 (99.1 to 99.3)
	RIFLE Failure		Sn=47.4 (44.2 to 50.6)
			Sp=97.6 (97.5 to 97.8)
			PPV=32.6 (30.2 to 35.2)
			NPV=98.7 (98.6 to 98.8)

All values are presented as percentages (%).

To convert serum creatinine from μmol/l to mg/dl divide by 88.4.

AKI, acute kidney injury; ICD-10, International Classification of Diseases, tenth revision; NPV, negative predictive value; PPV, positive predictive value; Sn, sensitivity; Sp, specificity; n/a, not applicable.

Sensitivities were higher at hospital admission than at presentation to the emergency department for all four serum creatinine-based definitions of AKI. For example, at hospital admission, the sensitivity of the code for the RIFLE Injury definition was 61.6% (95% CI 57.5% to 65.5%). The sensitivity of the code improved for more severe definitions of AKI, peaking at the RIFLE Injury definition. There was no substantial difference in specificity for both types of hospital encounters, with values greater than 95% in both settings. The positive-predictive value of the code decreased for more severe definitions of AKI, with a nadir at the RIFLE Injury definition in both settings.

The absolute change in serum creatinine (the peak value—the baseline value) and relative change in serum creatinine ((the peak value—the baseline value)/the baseline value)) for patients with hospital encounters who were positive and negative for the ICD-10 N17× code are presented in table 3. When considering the ‘all diagnoses’ algorithm, 1.2% of patients at presentation to the emergency department and 5.2% of patients at hospital admission were code positive for AKI.

Table 3

Change in serum creatinine concentration from the baseline in all patients with and without the ICD-10 N17× code for AKI (referred to as code positive and code negative)

		Emergency department cohort			Hospitalised cohort
			Absolute change (µmol/l)	Relative change (%)*		Absolute change (µmol/l)	Relative change (%)*
Diagnostic coding algorithm	Code	N	Median (IQR)		N	Median (IQR)
All diagnoses	+	426	133 (62 to 288)	87 (43 to 204)	2023	98 (43 to 200)	69 (28 to 153)
	−	35623	2 (−8 to 14)	2 (−9 to 15)	36 543	6 (−4 to 20)	7 (−5 to 22)
Main diagnosis/most responsible diagnosis	+	227	187 (89 to 383)	128 (62 to 295)	388	196 (93 to 396)	121 (49 to 275)
	−	35822	2 (−8 to 14)	2 (−9 to 16)	38 178	7 (−4 to 22)	7 (−4 to 24)
Admission diagnosis	+	n/a			1366	114 (39 to 187)	75 (30 to 169)
	−				37 230	6 (−4 to 21)	7 (−4 to 23)

Both absolute and relative changes in serum creatinine were significantly different between code positive and code negative patients in both types of hospital encounters (all p values <0.001) (means presented in box plot; online supplementary figures S2 and 3)

*((peak serum creatinine—baseline serum creatinine)/baseline serum creatinine)).

To convert serum creatinine from μmol/l to mg/dl divide by 88.4.

AKI, acute kidney injury; CKD, chronic kidney disease; ICD-10, International Classification of Diseases, tenth revision; N, number; +, code positive; −, code negative; n/a, not applicable.

The median (IQR) absolute change in serum creatinine concentration for code positive patients was 133 (62 to 288) μmol/l and 98 (43 to 200) μmol/l in each setting, respectively. The change for code negative patients was 2 (−8 to 14) μmol/l and 6 (−4 to 20) μmol/l in each setting, respectively.

When expressed in relative terms, the median (IQR) change for code positive patients was 87 (43 to 204)% and 69 (28 to 153)% in each setting, respectively. The relative change for code negative patients was 2 (−9 to 15)% and 7 (−5 to 22)% in each setting, respectively. In both settings, the difference in the mean absolute and relative change in serum creatinine between code-positive and code-negative patients was highly statistically significant (p<0.001).

The diagnostic performance of the ‘all diagnoses’ algorithm at hospital admission in patients with and without CKD is presented in table 4. The sensitivity of the ICD-10 code for AKIN Stage 1 or greater, RIFLE Risk and RIFLE Injury definitions were higher in patients with CKD than those without CKD. For example, the sensitivity of the code for the RIFLE Injury definition was 75.6% (95% CI 60.7% to 86.2%) in patients with CKD and 60.5% (95% CI 56.2% to 64.5%) in patients without CKD.

Table 4

Diagnostic performance characteristics of the ICD-10 N17x code in hospitalised patients with and without CKD using serum creatinine-based definitions of AKI as the reference standard*

	Diagnostic performance characteristics (95% CI)
Definition	Patients with CKD	Patients without CKD
AKIN Stage 1	Sn=35.6 (32.0 to 39.3)	Sn=20.4 (19.4 to 21.4)
	Sp=92.0 (90.2 to 93.5)	Sp=98.6 (98.4 to 98.7)
	PPV=74.4 (69.2 to 78.9)	PPV=74.2 (72.1 to 76.2)
	NPV=68.7 (66.1 to 71.1)	NPV=85.9 (85.5 to 86.3)
RIFLE Risk	Sn=76.9 (65.4 to 85.5)	Sn=54.8 (51.4 to 58.2)
	Sp=83.5 (81.6 to 85.3)	Sp=96.5 (96.3 to 96.7)
	PPV=16.2 (12.5 to 20.8)	PPV=26.2 (24.2 to 28.3)
	NPV=98.9 (98.1 to 99.3)	NPV=98.9 (98.8 to 99.1)
RIFLE Injury	Sn=75.6 (60.7 to 86.2)	Sn=60.5 (56.2 to 64.5)
	Sp=82.6 (80.7 to 84.4)	Sp=96.2 (96.0 to 96.4)
	PPV=10.1 (7.2 to 13.9)	PPV=18.5 (16.8 to 20.5)
	NPV=99.2 (98.6 to 99.6)	NPV=99.4 (99.3 to 99.5)
RIFLE Failure	Sn=48.4 (42.2 to 54.6)	Sn=63.1 (59.4 to 66.6)
	Sp=86.4 (84.5 to 88.1)	Sp=96.4 (96.3 to 96.6)
	PPV=38.6 (33.4 to 44.2)	PPV=24.8 (22.8 to 26.9)
	NPV=90.4 (88.7 to 91.9)	NPV=99.3 (99.2 to 99.4)

All values are presented as percentages (%).

*The ICD-10 N17× coding algorithm considered is all diagnoses.

AKI, acute kidney injury; CKD, Chronic kidney disease; ICD-10, International Classification of Diseases, tenth revision; NPV, negative predictive value; PPV, positive predictive value; Sn, sensitivity; Sp, specificity.

The code demonstrated the highest sensitivity for the RIFLE Risk definition in patients with CKD and the RIFLE Failure definition in patients without CKD. The specificities of the code were lower in patients with CKD than those without CKD for all four definitions. For example, the specificity of the code for the RIFLE Injury definition was 82.6% (95% CI 80.7% to 84.4%) in patients with CKD and 96.2% (95% CI 96.0% to 96.4%) in patients without CKD.

The absolute and relative changes in serum creatinine at hospital admission in patients with and without CKD who were code positive and code negative are presented in table 5. When considering the ‘all diagnoses’ algorithm, a total of 18.9% of patients with CKD and 4.6% of patients without CKD were code positive for AKI.

Table 5

Change in serum creatinine concentration from the baseline in hospitalised patients with and without CKD where ICD-10 code N17× did and did not indicate AKI (referred to as code positive and code negative)*

	Patients with CKD			Patients without CKD
		Absolute change (µmol/l)	Relative change (%)†		Absolute change (µmol/l)	Relative change (%)†
Code	N	Median (IQR)		N	Median(IQR)
+	308	108 (48 to 215)	53 (20 to 104)	1715	95 (43 to 197)	72 (29 to 161)
−	1324	16 (−8 to 51)	9 (−4 to 26)	35 219	6 (−4 to 19)	6 (−5 to 22)

Code positive and code negative patients were significantly different (all p-values <0.001).

Relative changes in patients with and without CKD were statistically different (p<0.0001).

*The ICD-10 N17× coding algorithm considered is all diagnoses.

†((peak serum creatinine—baseline serum creatinine)/baseline serum creatinine)).

To convert serum creatinine from μmol/l to mg/dl divide by 88.4.

CKD, chronic kidney disease; ICD-10, International Classification of Diseases, tenth revision; N, number; +, code positive; −, code negative; n/a, not applicable.

The median (IQR) absolute change in serum creatinine concentration in patients with CKD who were code positive was 108 (48 to 215) μmol/l and in patients without CKD who were code positive was 95 (43 to 197) μmol/l. The difference in the absolute change in serum creatinine between patients with and without CKD who were AKI code positive was not significantly different (p=0.910). The median (IQR) absolute change in patients with CKD who were code negative was 16 (−8 to 51) μmol/l and in patients without CKD who were code negative was 6 (−4 to 19) μmol/l.

When expressed in relative terms, the median (IQR) change in serum creatinine in patients with CKD who were code positive was significantly lower than in patients without CKD who were code positive (53 (20 to 104)% vs 72 (29 to 161)%; p<0.0001). The median (IQR) relative change in patients with CKD who were code negative was 9 (−4 to 26)% and in patients without CKD who were code negative was 6 (−5 to 22)%. For both patients with and without CKD, the difference in the mean absolute and relative changes in serum creatinine between code positive and negative patients was highly statistically significant (p<0.001).

Discussion

In this population-based retrospective validation study, we evaluated the diagnostic performance of ICD-10 code N17× for AKI. We discovered that the best performing coding algorithm both at presentation to the emergency department and at hospital admission was when the code was expressed as ‘all diagnoses’ (ie, when the code was indicated by any diagnosis type).

At hospital admission, the poorest performing coding algorithm was the ‘most responsible diagnosis’. A likely explanation for this is that AKI frequently presents as a complication of other conditions requiring hospital admission, such as acute myocardial infarction or sepsis.5 AKI is rarely the primary reason for hospital admission so would less likely be coded as the ‘most responsible diagnosis’.

The ICD-10 code demonstrated modest sensitivity for the serum creatinine-based definitions of AKI, indicating its inability to identify a portion of patients who experienced a clinically significant increase in serum creatinine. Of the patients who satisfied the RIFLE Injury definition of AKI from their increase in serum creatinine, 61.6 (95% CI 57.5 to 65.5)% of them were code positive AKI when the ICD-10 code was expressed as ‘all diagnoses’. However, the code demonstrated high specificity for all four serum creatinine-based definitions. Of the patients who did not satisfy the RIFLE Injury definition of AKI, 95.6 (95% CI 95.4 to 95.8)% of them were code negative for AKI. These characteristics of the ICD-10 AKI code are similar to those described in ICD-9 code validation studies that used serum creatinine-based definitions as the reference standard.8 10 21 22 The use of a serum creatinine-based definition of AKI as the reference standard is preferred over a chart review, recognising the latter have appeared in a number of previous validation studies.31–35

The ICD-10 code showed improved sensitivity for more severe definitions of AKI and was the highest for the RIFLE Injury definition. This characteristic of the ICD-10 code is similar to that of ICD-9 codes for AKI.21 A greater increase in serum creatinine reflects more severe AKI, is more likely to be documented in the medical chart and thus detected by coders.

The positive predictive value of the ICD-10 code decreased for more severe definitions of AKI and was the lowest for the RIFLE Injury definition. This fluctuation in the positive predictive values can in part be attributed to the varying prevalence of the four serum creatinine-based definitions.

At both types of hospital encounters, patients who were code positive had a significantly higher increase in serum creatinine than those who were code negative (p<0.001). In other words, the code does successfully differentiate between two groups of patients with and without distinct changes in serum creatinine.

We also assessed the diagnostic performance of the ICD-10 code in subgroups of patients with and without CKD prior to the hospital encounter. For all definitions, with the exception of RIFLE Failure, the code demonstrated higher sensitivity in patients with CKD than those without CKD. However, the specificity of the code was lower in patients with CKD than those without CKD (for all definitions of AKI). The latter finding suggests a portion of patients with stable CKD are misclassified as having AKI at their hospital encounter. For example, clinicians may not have access to patients’ baseline serum creatinine measurements, or may make an AKI diagnosis without investigating the baseline measurements. In such cases, an elevated serum creatinine concentration at hospital presentation that is no different from the baseline value may still be misdiagnosed as AKI.

Among patients with CKD, presence or absence of the ICD-10 AKI code was also able to differentiate between two groups of patients with distinct changes in their serum creatinine from the baseline value. Although the difference in mean absolute change in serum creatinine was no different in patients with and without CKD who had a positive AKI code, the relative change was lower in patients with CKD than those without CKD. This is consistent with physicians defining AKI more commonly in absolute rather than in relative terms. With a given absolute increase in serum creatinine, the relative increase in serum creatinine is lower in patients with CKD than those without CKD.

Our study has several strengths. To our knowledge, it is the first study to provide information on the diagnostic performance of the ICD-10 code for AKI using serum creatinine-based definitions of AKI as the reference standard.8 10 21 22 We assessed different diagnostic coding algorithms of the ICD-10 code at both presentation to the emergency department and at hospital admission. Moreover, we evaluated the diagnostic performance of the code in subgroups of patients with and without CKD prior to the hospital encounter. We studied patients from 12 hospitals across Southwestern Ontario with representation from both academic and community care centres. This helped minimise a selection bias. The large number of patients resulted in good precision for the estimates provided in the study.

Our study does have some limitations. We evaluated the validity of the ICD-10 code for AKI in patients 66 years of age and older. These findings should generalise well to elderly patients, a segment of the population at a high risk of AKI.11 36 The findings are also useful for pharmacoepidemiological studies using Ontario's healthcare administrative databases, where prescription information on Ontario residents aged 65 and older is available from the universal drug benefit plan. However, future validation studies in younger patients are needed. Moreover, we did not know the degree to which patients with AKI were symptomatic from diminished kidney function or the indication that prompted presentation to the emergency department or hospital admission.

It is important to acknowledge that for the definitions of AKI used in this study, we adapted the serum creatinine-based component of the AKIN and RIFLE classification systems. The AKIN and RIFLE classification systems recommend using both serum creatinine and urine output measurements in determining the presence and severity of AKI.6 7 In addition, it is recommended that the AKIN classification is applied only after an optimal state of hydration is achieved.6 However, urine output measurement and hydration status were not available in the data sources used in the study. In reality, the accuracy of bedside urine output measurement is notoriously poor outside of intensive care settings with an indwelling catheter. Nonetheless, the change in serum creatinine is a widely used measure of kidney function in clinical settings. Moreover, the serum creatinine-based component has been solely used to identify patients with AKI using the AKIN and RIFLE classification systems in previous studies.37 38

The median (IQR) period between the baseline serum creatinine measurements and the hospital encounter was 102 (41–204) days for patients who presented to the emergency department and 39 (16–128) days for patients admitted to the hospital. While these are reasonable baseline measurements, the AKIN and RIFLE classification systems require the change in serum creatinine to occur within 48 h and within 7 days, respectively.6 7 Although it is likely that serum creatinine changes occurred just prior to the hospital encounter, we cannot say this with complete certainty given the absence of available measurements during this period.

Finally, we could not examine the validity of outpatient claims for AKI in this study. However, the diagnostic performance of outpatient claims in our jurisdiction is notoriously poor. Nonetheless, emergency department and hospital inpatient records hold information on more severe forms of AKI, which are of particular interest to clinicians, researchers and policymakers. Moreover, we recognise that we did not capture those patients who may have had severe forms of AKI, but did not present to the emergency department or hospital, or those who presented, but did not have their serum creatinine measured. However, the latter situation is unlikely given that serum creatinine measurements are a standard laboratory test for most patients who present to a hospital encounter for acute medical care.

Conclusion

Although the use of healthcare administrative databases for clinical or healthcare services research has several merits, there are inherent limitations, including the accuracy with which certain conditions (ie, AKI) can be identified. The sensitivity of ICD-10 code N17× for AKI was limited, particularly for less-severe definitions. This results in an underestimation of the true incidence of the condition. Nonetheless, the presence or absence of the ICD-10 code successfully differentiates two groups of elderly patients with and without distinct increases in serum creatinine from baseline values at the time of a hospital encounter. The results from this study guide's judicious use of ICD-10 code N17× in future research using large healthcare administrative databases.