Accuracy of ICD-9-CM Codes by Hospital Characteristics and Stroke Severity: Paul Coverdell National Acute Stroke Program.

BACKGROUND: Epidemiological and health services research often use International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes to identify patients with clinical conditions in administrative databases. We determined whether there are systematic variations between stroke patient clinical diagnoses and ICD-9-CM codes, stratified by hospital characteristics and stroke severity. METHODS AND
RESULTS: We used the records of patients discharged from hospitals participating in the Paul Coverdell National Acute Stroke Program in 2013. Within this stroke-enriched cohort, we compared agreement between the attending physician's clinical diagnosis and principal ICD-9-CM code and determined whether disagreements varied by hospital characteristics (presence of a stroke unit, stroke team, number of hospital beds, and hospital location). For patients with a documented National Institutes of Health Stroke Scale score at admission, we assessed whether diagnostic agreement varied by stroke severity. Agreement was generally high (>89%); differences between the physician diagnosis and ICD-9-CM codes were primarily attributed to discordance between ischemic stroke and transient ischemic attack (TIA), and subarachnoid and intracerebral hemorrhage. Agreement was higher for patients in metropolitan hospitals with stroke units, stroke teams, and >200 beds (all P<0.001). Agreement was lowest (60.3%) for rural hospitals with ≤200 beds and without stroke units or teams. Agreement was also lower for milder (94.9%) versus more-severe (96.4%) ischemic strokes (P<0.001).
CONCLUSIONS: We identified disagreements in stroke/TIA coding by hospital characteristics and stroke severity, particularly for milder ischemic strokes. Such systematic variations in ICD-9-CM coding practices can affect stroke case identification in epidemiological studies and may have implications for hospital-level quality metrics.

Introduction

Stroke is a leading cause of hospitalization and serious long‐term disability in the United States, with stroke‐related care and associated lost productivity resulting in considerable direct and indirect costs to individuals and society.1 Disease surveillance and quality‐of‐care studies frequently use information from administrative claims databases, such as Medicare, by identifying patients with specific conditions based on International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes. Studies comparing the accuracy of ICD‐9‐CM codes with medical record review find variations in agreement depending on code selection,2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18 as well as patient characteristics, such as age, race/ethnicity,10, 11 sex,11 and length of hospitalization.3 Aside from teaching hospital status10 and hospital departments,16 potential variations in accuracy based on other hospital characteristics are largely unknown. Such variations in agreement of ICD‐9‐CM codes with recorded clinical diagnoses could affect the interpretation of results for studies comparing quality of care and costs across care settings based on ICD‐9‐CM code selection.

We used data from the Paul Coverdell National Acute Stroke Program (PCNASP) to determine overall agreement between clinical diagnoses made by the attending physician and the principal ICD‐9‐CM billing code. Using this stroke‐enriched cohort, we described disagreements for stroke and transient ischemic attack (TIA) diagnoses. We also sought to determine whether there are systematic variations in agreement by hospital characteristics (presence or absence of a stroke unit and stroke team, number of hospital beds, and hospital location [metropolitan, micropolitan, or small town/rural]), in addition to stroke severity.

Methods

Data Source and Study Sample

The PCNASP was established in 2001 by the Centers for Disease Control and Prevention (CDC) to support state‐based acute stroke quality‐of‐care registries and activities to decrease rates of premature death and disability from stroke.19 The design of the PCNASP and data elements are published in detail elsewhere.20, 21 Briefly, the PCNASP collects both the clinical diagnosis as recorded in the medical record notes and the principal discharge ICD‐9‐CM code. Participating hospitals are instructed to abstract the final clinical diagnosis documented by the physician. Case identification is based on the final clinical diagnosis, and not ICD‐9‐CM codes. Hospitals are instructed to include patients 18 years and older for the following clinical diagnoses documented by the physician: ischemic stroke (IS), TIA, intracerebral hemorrhage (ICH), subarachnoid hemorrhage (SAH), and stroke type unspecified. Participating hospitals are encouraged to develop methods to ensure completeness of case ascertainment.21

For this analysis, we used PCNASP deidentified records for patients discharged from January 1, 2013 through December 31, 2013. There were 11 states participating in the PCNASP during this time period.

The ICD‐9‐CM code definitions for stroke and TIA were based on the American Heart Association/American Stroke Association (AHA/ASA) updated definition of stroke, but excluded retinal and spinal infarction and included V12.54 for TIA.22 Based on these definitions, ICD‐9‐CM codes were categorized as (1) IS (433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.01, 434.11, and 434.91); (2) TIA (435.0, 435.1, 435.2, 435.3, 435.8, 435.9, and V12.54); (3) SAH (430); or (4) ICH (431). Additionally, ICD‐9‐CM code 436 was defined as strokes that were not otherwise specified. The final cohort for all analyses was restricted to patients with a single clinical diagnosis documented by the attending physician (Figure). Records were excluded if there was a missing ICD‐9‐CM code. Additionally, we excluded patients with in‐hospital strokes because they were likely admitted for a different reason, which could result in a principal ICD‐9‐CM code unrelated to stroke.

Figure 1

Patient population for primary and secondary analyses. Patient record flow diagram of how the final sample size was determined for the primary agreement (concordance) analysis and the secondary hospital and stroke severity analysis. ICD‐9‐CM indicates International Classification of Diseases, Ninth Revision, Clinical Modification; NIHSS, National Institutes of Health Stroke Scale.

This study and waiver for informed consent was approved by the CDC's Institutional Review Board.

Hospital Characteristics and Stroke Severity

Hospital characteristics were self‐reported by PCNASP hospitals to state departments of health and included the presence of a stroke unit, presence of a stroke team, category of number of hospital beds, and hospital location. Stroke units were defined as being staffed and directed by personnel (eg, physicians, nurses, and speech therapists) with training and expertise in caring for patients with cerebrovascular disease, and did not necessarily have to be distinct wards or units. Stroke teams were defined as including a physician with expertise in diagnosing and treating cerebrovascular disease, as well as an additional health care provider (eg, nurse). The stroke team had to be available 24 hours per day and to see patients within 15 minutes of being called. Consistent with recent literature, the number of hospital beds was categorized as 0 to 100, 101 to 200, 201 to 300, 301 to 500, or ≥501 beds.23 A hospital's location was defined using Rural‐Urban Commuting Area codes, which were further categorized using a mechanism by the United States Department of Agriculture Economic Research Service (codes 1–3 were metropolitan, 4–6 were micropolitan, and ≥7 were small town/rural areas).24 The National Institutes of Health Stroke Scale (NIHSS) score at admission was used as a measure of stroke severity and was divided into 4 categories: 0 to 7, 8 to 13, 14 to 21, and 22 to 42, with higher categories indicating more‐severe strokes.25

Statistical Analysis

The primary analysis evaluated the agreement (concordance) between the clinical diagnosis recorded in the medical record notes by the attending physician and the principal ICD‐9‐CM billing code recorded within PCNASP patient records. Concordance for IS was calculated as the number of records with both a clinical diagnosis and principal ICD‐9‐CM code for IS, compared with the total number of records having either a clinical diagnosis or ICD‐9‐CM code for IS. A similar approach was used to calculate concordance for TIA, ICH, and SAH diagnoses. The kappa coefficient was also calculated for each category. We identified the most frequent ICD‐9‐CM codes and the attending physician's clinical diagnoses for discordant results.

For the secondary analysis of agreement by hospital characteristics, we restricted the cohort to hospitals that provided hospital infrastructure data. Additionally, the secondary analysis of stroke severity was restricted to records with a documented NIHSS score at hospital admission.

Chi‐square tests and Fisher's exact tests were used to compare agreement based on hospital characteristics and levels of stroke severity, with a P<0.05 considered to be statistically significant. SAS software (version 9.3; SAS Institute Inc., Cary, NC) was used for statistical analyses.

Results

There were 90 035 patient records from 11 states participating in the PCNASP, which included 371 hospitals in 2013. The final sample for the primary analysis included 85 024 records (mean age, 69.6±14.9 years; 48.1% men; 74.5% white; 23.5% with a past stroke; 9.3% with a past TIA; Figure; Table 1).

Table 1

Characteristics of Included Patients

Characteristic	Primary Analysis: Agreement (Concordance) (N=85 024)a	Secondary Analysis: Hospital Characteristics (N=67 442)a	Secondary Analysis: Stroke Severity (N=55 373)a
Age, y (mean±SD)	69.6±14.9	69.5±15.0	70.2±14.7
Sex, n (%)
Male	40 846 (48.1)	32 563 (48.3)	26 927 (48.6)
Female	44 059 (51.9)	34 873 (51.7)	28 451 (51.4)
Race/ethnicity, n (%)
White	62 940 (74.5)	49 418 (73.5)	41 777 (75.6)
Black	15 966 (18.9)	13 597 (20.2)	9617 (17.4)
Other	5575 (6.6)	4222 (6.3)	3833 (6.9)
NIHSS, median (range)	3 (0–42)	3 (0–42)	3 (0–42)
Medical history, n (%)
Hypertension	61 722 (72.6)	49 575 (73.5)	41 835 (75.5)
Dyslipidemia	37 330 (43.9)	30 135 (44.7)	25 743 (46.5)
MI or CAD	19 902 (23.4)	16 077 (23.8)	13 423 (24.2)
Heart failure	7670 (9.0)	6215 (9.2)	5063 (9.1)
Diabetes	25 476 (30.0)	20 372 (30.2)	17 048 (30.8)
Atrial fibrillation	14 707 (17.3)	11 625 (17.2)	10 102 (18.2)
Prior stroke	20 004 (23.5)	16 020 (23.8)	13 365 (24.1)
Past TIA/VBI	7925 (9.3)	6518 (9.7)	5636 (10.2)
Smoking	15 951 (18.8)	13 025 (19.3)	10 280 (18.6)

CAD indicates coronary artery disease; NIHSS, National Institutes of Health Stroke Scale; MI, myocardial infarction; TIA, past transient ischemic attack; VBI, vertebrobasilar insufficiency.

Numbers may not sum to totals because of missing data.

The kappa coefficient ranged from 0.91 to 0.96 for the diagnostic categories (Table 2). Additionally, simple agreement was >89% for each of the 4 stroke/TIA diagnostic categories and was highest for IS, followed by SAH, TIA, and ICH (Table 2). For IS, the attending physician diagnosis and the principal ICD‐9‐CM codes agreed for 94.1% of records. The ICD‐9‐CM codes for discordant records were commonly stroke related (admission for carotid occlusion, TIA, or stroke not otherwise specified instead of IS). For TIA, 91.4% of records were concordant; discordances were most frequently attributed to a recorded admission for carotid occlusion or IS. Agreement was 93% for SAH and 89.4% for ICH.

Table 2

Agreement and Discordance by Stroke/TIA Diagnoses

Category	Diagnosis		% Diagnosis Category (n)	Kappa Coefficient (SE)	Frequency of Discordance Explanations (n)
	Clinical Diagnosis	ICD‐9‐CM Code
ISa	IS	IS	94.1 (55 794)	0.91 (0.0016)	N/A
	IS	Not IS	3.6 (2136)		Carotid artery occl. 433.10 (989); TIA (363); SNS (222); Cerebral artery occl. 434.90 (68)
	Not IS	IS	2.3 (1357)		SNS (892); TIA (251); ICH (127); NoS (77)
TIAb	TIA	TIA	91.4 (12 566)	0.95 (0.0015)	N/A
	TIA	Not TIA	5.3 (734)		Carotid artery occl. 433.10 (282); IS (243)
	Not TIA	TIA	3.3 (456)		IS (385); NoS (36); SNS (29); SAH (5)
SAHc	SAH	SAH	93.0 (3265)	0.96 (0.0024)	N/A
	SAH	Not SAH	4.4 (153)		ICH (64); Subdural hemorrhage‐432.10 (28); Intracranial hemorrhage‐432.90 (18); IS (7)
	Not SAH	SAH	2.6 (93)		ICH (79); IS (11); NoS (2); SNS (1)
ICHd	ICH	ICH	89.4 (8130)	0.94 (0.002)	N/A
	ICH	Not ICH	9.4 (859)		Intracranial hemorrhage‐432.90 (527); IS (95); SAH (79); Subdural hemorrhage‐432.10 (27)
	Not ICH	ICH	1.2 (110)		SAH (64); IS (36); SNS (5); NoS (4)

ICD‐9‐CM indicates International Classification of Diseases, Ninth Revision, Clinical Modification; ICH, intracerebral hemorrhage; IS, ischemic stroke; N/A, not applicable; NoS, no stroke‐related diagnosis; occl., occlusion; SAH, subarachnoid hemorrhage; SNS, stroke not otherwise specified; TIA, transient ischemic attack.

ICD‐9‐CM codes 433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.01, 434.11, and 434.91.

ICD‐9‐CM codes 435.0, 435.1, 435.2, 435.3, 435.8, 435.9, and V12.54.

ICD‐9‐CM code 430.

ICD‐9‐CM code 431.

Hospital characteristic data were available in 7 states and included 255 hospitals, which yielded a sample size of 67 442 patient records (mean age, 69.5±15.0 years; 48.3% men; 73.5% white; 23.8% with a past stroke; 9.7% with a past TIA; Figure; Table 1). Agreement between a clinical diagnosis of IS and an ICD‐9‐CM code for ischemic stroke was higher for hospitals with designated stroke units, stroke teams, larger numbers of beds, and locations in metropolitan areas (Table 3). For example, for those with IS, 94.6% of records were concordant for hospitals with a stroke unit compared with 86.7% for hospitals without a stroke unit (P<0.001). Metropolitan hospitals with >200 beds as well as a dedicated stroke unit and team (N=24 hospitals, treating 11 047 ischemic stroke patients) were associated with the highest agreement between the clinical diagnosis and an ICD‐9‐CM code for IS (96.9%, data not shown). In contrast, IS record concordance was lowest for hospitals that were located in rural areas, had ≤200 beds, and did not have a stroke unit or team (N=9 hospitals, treating 94 IS patients; 60.3%, data not shown). Similarly, agreement was higher for TIA for hospitals with designated stroke units, larger numbers of beds, and locations in metropolitan areas. For SAH and ICH, better agreement was also found for larger hospitals.

Table 3

Agreement by Hospital Characteristics

	Ischemic Stroke % (n)	TIA % (n)	Subarachnoid Hemorrhage % (n)	Intracerebral Hemorrhage % (n)
Stroke unit (%)a
Yes	94.6 (34 249)b	92.0 (7718)b	94.4 (2399)	89.3 (5541)b
No	86.7 (2855)	85.1 (811)	95.0 (94)	84.9 (662)
Stroke team (%)c
Yes	94.5 (32 678)b	91.4 (7395)	94.4 (2354)	89.3 (5390)
No	89.7 (4436)	91.0 (1196)	95.7 (134)	88.4 (404)
No. of beds (%)d
0 to 100	80.8 (1422)b	79.1 (569)b	25.0 (1)b	69.4 (50)b
101 to 200	94.1 (5006)	92.2 (1538)	90.8 (59)	83.1 (330)
201 to 300	89.2 (6446)	91.7 (1821)	91.4 (222)	81.4 (804)
301 to 500	96.3 (11 974)	93.1 (3713)	90.9 (371)	90.6 (1541)
501+	95.8 (18 833)	91.6 (2870)	95.5 (1991)	91.0 (3613)
Location (%)e
Metropolitan	94.8 (24 303)b	92.3 (4892)b	96.5 (1682)b	91.0 (3965)
Micropolitan	86.9 (2082)	88.8 (501)	92.7 (89)	88.7 (236)
Rural	91.7 (1144)	89.2 (297)	90.5 (57)	93.5 (157)

TIA indicates transient ischemic attack.

Includes 196 hospitals.

Statistically significant at P<0.05.

Includes 199 hospitals.

Includes 252 hospitals.

Includes 140 hospitals.

In the secondary analyses of agreement based on stroke severity, we restricted the cohort to 55 373 patient records with a documented NIHSS score at hospital admission (mean age, 70.2±14.7 years; 48.6% men; 75.6% white; 24.1% with a past stroke; 10.2% with a past TIA; Figure; Table 1). Agreement was lower for mild ischemic strokes (NIHSS 0–7) compared with moderate and severe strokes (NIHSS ≥8; P<0.001; Table 4). Discordant diagnoses for mild ISs included carotid occlusion, TIA, and stroke not otherwise specified. Compared to TIAs with NIHSS <8, we found that patients with a clinical diagnosis of TIA and NIHSS ≥8 more commonly had a documented previous stroke (23.4% vs 44.5%) or were less able to ambulate with or without a device before the current event (1.8% vs 11.7%).

Table 4

Agreement by NIHSS at Admission

NIHSS Category	Ischemic Strokea	TIAa
0 to 7	94.9 (27 139)	92.6 (8792)
8 to 13	96.4 (5119)	92.3 (395)
14 to 21	96.4 (4032)	87.8 (130)
22 to 42	96.3 (2439)	82.5 (33)

NIHSS indicates National Institutes of Health Stroke Scale; TIA, transient ischemic attack.

Statistically significant at P<0.05.

Discussion

In this study of 85 024 patients from more than 300 hospitals participating in the PCNASP, we found disagreement between the clinical diagnoses and recorded primary ICD‐9‐CM codes for the 4 primary stroke/TIA diagnostic categories that ranged from 5.9% to 10.6%. Agreement rates varied by hospital characteristics, with higher rates in hospitals with stroke units and teams, and in larger metropolitan hospitals as compared with hospitals without these characteristics. In analyses stratified by NIHSS, we noted disagreement between TIAs and milder strokes.

As noted in a recent AHA/ASA statement on risk adjustment of IS outcomes for comparing hospital performance, there is a need to study the consistency of coding across sites for key prognostic variables commonly used in risk‐adjustment models.26 Systematic variations in case ascertainment could impact the generalizability of measures developed to assess hospital performance and reimbursement based on ICD‐9‐CM discharge codes. We noted higher agreement between the clinical diagnosis and primary discharge code in hospitals with dedicated stroke units and teams. This supports the Brain Attack Coalition's premise that stroke units and teams are associated with better patient care processes,27 in part attributed to the availability of subspecialists, and imaging resources that improve the diagnostic classification of conditions based on current guidelines.28 The observed disagreement between TIAs and minor strokes in our study warrants further investigation in future studies, given the forthcoming inclusion of the NIHSS to assess severity in ICD‐10‐CM codes.

This study has several limitations. Because the PCNASP is a quality improvement program, results may not be generalizable to hospitals that are not enrolled in this or a similar program. There may be unmeasured variability in methods used by PCNASP hospitals to identify cases and assign diagnoses that drive patterns of inaccuracy by hospital characteristics. For example, hospitals using a 2‐stage system that has the physician assign a billing code that is later confirmed by administrative personnel may differ from a system that only has a billing coder collect the information. There may also be unmeasured residual correlation attributed to clustering of patients within hospitals. We are unable to directly assess the role of the use of electronic medical records, but we used standardized criteria that is consistent with the approach used by Get With The Guideline‐Stroke data abstraction, and hospitals participating in the PCNASP are required to have a sample of records reabstracted each year to assure the reliability of the data across sites. Because data on all 4 hospital characteristics and stroke severity were only available on a subset of records, we did not conduct multivariable analysis to identify factors independently associated with agreement. We could not assess agreement for the small proportion of records (4.3%; 3867 of 90 035) excluded because of a missing clinical diagnosis or ICD‐9‐CM codes.

When using and interpreting results from studies that use administrative data, researchers, policy makers, and public quality reports should be aware of the systematic differences we found in the accuracy of ICD‐9‐CM codes. Variations in the accuracy of ICD‐9‐CM stroke codes by hospital characteristics may affect comparative assessments of performance and has implications for hospital reimbursements.

Disclosures

None.