Positive predictive value of ICD-10 diagnosis codes for brain abscess in the Danish National Patient Registry.

PURPOSE: To evaluate the positive predictive value (PPV) of ICD-10 diagnosis codes for brain abscess in the Danish National Patient Registry (DNPR). PATIENTS AND METHODS: We examined medical records of all patients with a first-time diagnosis code of brain abscess in the DNPR from 2007 to 2016. Patients were categorized with either confirmed or disproved brain abscess using a priori defined criteria. We computed the PPV as the proportion of confirmed diagnoses.
RESULTS: We identified 709 patients, of whom the medical records could not be retrieved for 15 patients, leaving 694 for further analyses. Of these, 444 had a confirmed brain abscess and 250 had another proven diagnosis, including 47 cases of intracranial empyema. The overall PPV was 64% (95% CI: 60-68) ranging from 24% to 96% among the different codes evaluated. By including only patients with either 1) both a diagnosis and surgical procedure code for brain abscess or 2) patients admitted to hospital with certain primary diagnosis codes (DG060[C,E,F] or DG079B) without newly diagnosed central nervous system (CNS) cancer, spondylodiscitis/intraspinal abscess, or procedure codes for evacuation of intracranial empyema, the PPV increased to 84% (95% CI 80-87) and 89% (395/444) of all confirmed cases were identified.
CONCLUSION: The overall PPV of diagnosis codes for brain abscess in the DNPR was moderate. However, by exclusion of newly diagnosed CNS tumors, spondylodiscitis/intraspinal abscess, and intracranial empyemas, the PPV was high and therefore suitable for future registry-based studies of brain abscess.

Introduction

Brain abscess is a rare but life-threatening infection of the central nervous system (CNS) with a yearly incidence ranging from 0.2 to 1.3 per 100,000 population and a case-fatality rate of 5%–10%.1,2 Previous reports of the disease consist almost exclusively of medical record reviews from single-center tertiary care facilities. Accordingly, epidemiological studies of incidence, risk factors, and long-term outcome of brain abscess are scarce. Health care registries are readily available at low costs and may be advantageous for examining such rare conditions on a large scale. One previous study used ICD version 9 (ICD-9) codes for brain abscess in a national health insurance research database in Taiwan to examine temporal changes in incidence and prognostic factors of brain abscess.3 However, the diagnostic codes were not validated and the overall incidence of 1.9 per 100,000 per year exceeded estimates from other studies.4–7 Hence, the incidence rate may be overestimated due to inclusion of patients with other diagnoses.

Government-funded universal health care combined with nationwide population-based health registries and unique linkage on an individual level provides researchers in Denmark with an excellent opportunity to examine both common and rare diseases from an epidemiological perspective.8 However, a high quality of data is important for research based on health registries, and the validity of diagnosis codes for brain abscess is unknown. Therefore, we examined the positive predictive value (PPV) of ICD-10 diagnosis codes for brain abscess in the Danish National Patient Registry (DNPR).

Methods

Setting and study population

We conducted this nationwide population-based cross-sectional validation study by reviewing the hospital records of all patients in Denmark with a first-time ICD-10 diagnosis code of brain abscess in the DNPR from January 1, 2007 to December 31, 2016. In the fourth quarter of 2016, the population of Denmark was 5,745,526.9 For Danish residents, all hospital care is tax-funded and thereby free of charge at the point of delivery. According to Danish legislation, medical records are to be stored for at least 10 years. All Danish citizens are given a unique civil registration number (CRN) at birth or immigration until death or emigration.10 The CRN enables unambiguous linkage to multiple Danish registries, including the DNPR.

The Danish National Patient Registry

Since the establishment in 1977 by the Danish Board of Health, this registry contains information on all somatic hospital admissions in Denmark, including primary, secondary, and supplementary diagnosis codes, surgical procedure codes, hospital and department codes, admission type, referral information, and time of admission and discharge.11 As of 1995, all somatic outpatient contacts as well as all emergency department and psychiatric contacts (inpatient and outpatient) have also been included. ICD-8 was used for classification of diagnoses up through 1993 and ICD-10 since then.

At discharge, the attending physician assigns a primary discharge diagnosis code to conclude the hospital contact (the main reason for hospitalization) and may optionally include up to 20 secondary and supplementary diagnosis codes to describe other conditions of relevance. The codes are sent electronically to the DNPR on a daily basis.

Brain abscess codes (ICD-10)

We included all patients registered with a first-time primary, secondary, or supplementary ICD-10 diagnosis code for brain abscess during the study period (Please see appendix for codes used).

Record review and definition of brain abscess

First, we confirmed the CRN and categorized dates of admission for these first-time hospital contacts as index dates. Second, we reviewed the medical records, including doctor’s notes, diagnostic imaging as described by the radiologist, microbiological results, and outcomes of the patients (NK and JB). In cases of doubt, categorizations of patients were resolved by discussion. We classified patients according to the following criteria:

Definite brain abscess was defined according to either of the following criteria:

Microbiologically confirmed brain abscess by aspiration and/or excision.

Aspirated pus or excised brain abscess and a clinical picture consistent with this disease, but microbiological investigations remained negative.

Clinical characteristics and imaging results suggested brain abscess combined with bacteremia or identification of another relevant pathogen from a normally sterile sample site (eg, pulmonary aspergillosis or nocardiosis), but no diagnostic neurosurgical procedure performed.

Probable brain abscess was defined as clinical characteristics and imaging results suggested brain abscess in combination with a favorable response to antimicrobial treatment, but microbiological investigations remained negative and no diagnostic neurosurgical procedures were performed.

Possible brain abscess was defined as clinical characteristics and imaging results suggested brain abscess, but patients passed away or were discharged before further examinations had been performed or a favorable response to antibiotic treatment could be observed.

Brain abscess was ruled out when the patient did not fulfill the criteria described previously or had proven alternative diagnosis, such as cancer without evidence of secondary brain abscess. This also included patients who survived without neurosurgical or antibiotic treatment.

Statistical analyses

The PPV was defined as the proportion of confirmed cases using record review as the reference standard. The proportion of cases identified by various criteria out of all cases confirmed by record review of the total cohort was also assessed.

First, we computed the PPV and 95% CIs of the diagnosis codes by grouping definite, probable, and possible brain abscess as “confirmed brain abscess.” We also performed stratified analyses by age groups, sex, primary vs secondary/supplementary diagnosis codes, and type of hospital contact (inpatients vs outpatients).

Next, we computed the PPV using only patients with either 1) both diagnosis and procedure code of brain abscess (aspiration or excision) or 2) inpatients with certain primary diagnosis codes (DG060[C,E,F] or DG079B) without: 1) a first-time primary diagnosis code for CNS tumor within 90 days after the index date for brain abscess, 2) a first-time primary diagnosis code for spondylodiscitis/intraspinal abscess 90 days before or after the index date, or 3) a surgical procedure code for evacuation of intracranial empyema. Diagnosis codes used for these conditions are listed in the appendix (Table S1).

Study data were collected and managed using REDCap electronic data capture tools hosted at North Denmark Region.12 We used STATA/MP® version 15 (StataCorp LLC, College Station, TX, USA) for all statistical analyses.

Ethical considerations

The study was approved by the Danish Data Protection Agency (record number 2008-58-0028) and the Danish Board of Health (record number 3-3013-1873/1/). Patient consent or permission from an ethical committee is not required for this type of study in Denmark.

Results

We identified 709 patients with a first-time diagnosis code for brain abscess during the study period. The medical records could not be found for 15 patients, leaving 694 patients for further analyses. The diagnosis was confirmed in 444 patients yielding an overall PPV of 64% (95% CI: 60–68) (Table 1). Among the 250 patients with disproved brain abscess, 47 had intracranial empyemas. The number of confirmed cases (range 1–317) and PPVs (range 24%–96%) varied greatly among the different ICD-10 brain abscess codes (Table 2). In subgroup analyses, we observed a higher PPV for patients identified by primary diagnosis codes (70%; 95% CI: 66–74) compared with patients included solely by secondary/supplementary diagnosis codes (28%; 95% CI: 20–38) and for inpatient contacts (67%; 95% CI: 63–70) compared with patients included solely by outpatient contacts (21%; 95% CI: 10–36) (Table 3). Similarly, the PPVs were substantially lower for age groups 0–18, 19–30, and >71 years of age compared with that for patients of 51–70 years of age (Table 3). For patients admitted at neurosurgical departments with a diagnosis code for brain abscess and a surgical procedure code for aspiration or excision of brain abscess, the PPV was 96% (95% CI: 92–98) while 241/444 (54%) of all cases were identified.

Table 1

Results of medical record review of all patients assigned with ICD-10 brain diagnosis codes in the Danish National Patient Registry, 2007–2016

Brain abscess validation	Number of cases
Definite	389
Probable	41
Possible	14
Confirmed	444
Disproved	250

Table 2

List of ICD-10 diagnosis codes for brain abscess in the Danish National Patient Registry and corresponding positive predictive values

ICD-10	Description	Cases confirmed/ases identifieda	PPV (95% CI)
DG060	Intracranial abscess and granuloma	308/449	69 (64–73)
DG060C	Otogenic brain abscess	3/11	27 (6–61)
DG060E	Cerebellar abscess	22/23	96 (78–100)
DG060F	Cerebral abscess	317/393	81 (76–84)
DG079	Intracranial and intraspinal abscess and granuloma in diseases classified elsewhere	11/45	24 (13–40)
DG079B	Intracranial abscess in disease classified elsewhere	10/14	71 (42–92)
DA066	Amoebic brain abscess	2/6	33 (4–78)
DB431	Phaeomycotic brain abscess	1/3	33 (1–91)

Notes:

Some patients had several hospital contacts with different ICD-10 codes of brain abscess.

Abbreviation: PPV, positive predictive value.

Table 3

Positive predictive values of ICD-10 codes for brain abscess by subgroups of patients in the Danish National Patient Registry

Characteristic	Cases confirmed / cases identified	PPV % (95% CI)
Overall	444/694	64 (60–68)
Male	277/425	65 (60–70)
Female	167/269	62 (56–68)
Inpatient	435/651	67 (63–70)
Outpatient	9/43	21 (10–36)
Primary diagnosis code	415/592	70 (66–74)
Secondary diagnosis codes	29/102	28 (20–38)
Neurosurgical departments	323/372	87 (83–90)
Neurosurgical departments + aspiration/excision	241/256	96 (92–98)
Infectious diseases departments	223/267	84 (79–88)
0–18 years	30/61	49 (36–62)
19–30	21/40	53 (36–68)
31–50	114/166	69 (61–76)
51–70	201/293	69 (63–74)
71–	78/134	58 (49–67)
Modified algorithma	395/470	84 (80–87)

Notes: 1) A brain abscess diagnosis code and a procedure code or 2) inpatients with primary diagnosis codes DG060(C,E,F) or DG079B without central nervous system tumor or spondylodiscitis diagnosis codes around the time of the index date for brain abscess or a surgical procedure code for evacuation of intracranial empyema.

The algorithm refers to patients with either (1) or (2).

Abbreviation: PPV, positive predictive value.

The diagnosis codes DA066 (n=6) and DB431 (n=3) were only rarely used and DG079 (n=45) consisted primarily of intraspinal abscesses. Therefore, we conducted a secondary analysis, including only patients with either 1) a brain abscess diagnosis code and a procedure code or 2) inpatients with primary diagnosis codes DG060(C,E,F) or DG079B without CNS tumor or spondylodiscitis primary diagnosis codes around the time of the index date for brain abscess or a surgical procedure code for evacuation of intracranial empyema. Using these criteria, the PPV increased to 84% (95% CI: 80–87) and 395/444 (89%) of all confirmed cases were identified. There were no differences in PPV between age groups in this analysis (data not shown).

The list of diagnoses among 75 patients with disproved brain abscess using these criteria included 11 cases (15%) of malignant tumor/metastasis, 11 cases (15%) of intracranial empyemas, seven cases (9%) of surgical site infections after neurosurgery (not intracranial abscess or empyema), five cases (7%) of dental infection without brain abscess, and three cases (4%) of bacterial meningitis; the remaining 38 cases had various other conditions.

Discussion

We found a crude PPV of 64% of ICD-10 codes for brain abscess in the DNPR, which increased to 84% by exclusion of patients with procedure codes for intracranial empyema and first-time diagnosis codes for CNS tumors and spondylodiscitis.

Patients with brain abscess often present with unspecific symptoms such as headache (69%), fever (53%), or neurological deficits (48%).1 Based on our record review, such patients were often admitted at emergency departments or neurological wards until cranial imaging revealed a mass lesion and subsequently, the main differential diagnoses were either tumor or abscess. Before transfer to the neurosurgery department for biopsy, the physician on duty assigned a diagnosis code of brain abscess for the hospital contact. However, in some cases, the biopsy revealed CNS tumor or metastasis. By excluding patients with a new ICD-10 code for CNS cancer within 90 days after the index date for brain abscess, we were able to substantially increase the PPV for brain abscess codes. Moreover, DG079 “Intracranial or intraspinal abscess” was primarily used for patients with spinal abscesses and by exclusion of these patients as well as patients with surgical procedure codes for intracranial empyema, the PPV increased further. Although we are not aware of a specific cut-off that establishes a PPV as suitable for epidemiological investigations, numerous other validation studies of diagnosis codes in the DNPR have found PPVs in similar ranges and considered them useful to this purpose.11

In comparison, most previous epidemiological studies of brain abscess are tertiary-center studies from departments of neurosurgery and they do not provide detailed descriptions of how patients were identified.5–7,13,14 The only study to use diagnosis codes in a national health registry (an insurance research database in Taiwan) was based on ICD-9 codes and the codes were not validated.3 A Danish study used ICD-10 codes to retrieve cases treated for brain abscess at three departments at their center, but did not report which codes were used or their accuracy.4 Another Danish study used surgical procedure codes for aspiration of brain abscess (KAAM00) and identified 90 patients. Among these, 80 were confirmed to have brain abscess after review of medical records, yielding a PPV of 89% (95% CI: 80–94), similar to our PPV of 84% using the previously described criteria.15 However, in our nationwide population-based study, we observed that only 54% of patients with brain abscess would have been identified by retrieval of ICD-10 diagnosis and procedure codes for brain abscess at departments of neurosurgery. If these results also apply to other countries, this suggests that previous estimates of incidence and descriptive studies of brain abscess may be incomplete. The observed differences between PPVs in different age groups were not found in our secondary analysis and were likely attributable to relatively higher incidences of cancer and subdural empyemas in these age groups compared with the reference age group.

Keeping the overall results of both the present and previous validation studies in mind, researchers using administrative databases for epidemiological research should carefully consider validating the diagnosis codes to ensure more accurate interpretations of their results. Specifically, the algorithm used in this study may be helpful to researchers in other countries performing epidemiological studies of brain abscess using ICD codes from administrative databases.

We used the administrative DNPR to identify all patients with a hospital contact assigned a primary, secondary, or supplementary ICD-10 diagnosis code of brain abscess in Denmark from 2007 to 2016. There were no major changes in clinical practice or accessibility to diagnostic or therapeutic modalities in Denmark during the study period. Additionally, the medical records could be retrieved in all but 15 cases, leaving the dataset nearly complete. For these reasons, we expect selection bias to be limited. As the researchers evaluating the medical records in this study knew the diagnosis code beforehand, ascertainment bias is a risk, which may favor a confirmation of the diagnosis code in cases of doubt. However, we accessed all available information in the medical records and used a strict set of definitions to confirm or disprove the diagnosis, which were defined prior to the search. We were not able to estimate sensitivity, specificity, or negative predictive values of ICD-10 codes for brain abscess in the DNPR, because we only examined patients with a diagnosis code for brain abscess. Nonetheless, if we assume an unfavorable scenario with a modest sensitivity in our setting with a low prevalence, the PPV is a fair approximation of specificity and may as such be used for unbiased relative measures of risk.16

Conclusion

The overall PPVs of diagnosis codes for brain abscess in the DNPR were moderate. However, after exclusion of newly diagnosed CNS tumors, spondylodiscitis/intraspinal abscess, and intracranial empyemas, the PPV was high and may be suitable for future registry-based studies of brain abscess.

Supplementary materials

The Danish Study Group of Infections of the Brain (DASGIB)

JB, Department of Infectious Diseases, Aalborg University Hospital, Aalborg, Denmark; MS, Department of Infectious Diseases, Aarhus University Hospital Skejby, Aarhus, Denmark; LL, Department of Infectious Diseases, Odense University Hospital, Odense, Denmark; LW, Department of Pulmonary and Infectious Diseases, Sjællands University Hospital Roskilde, Roskilde, Denmark; JHL, Department of Infectious Diseases, Rigshospitalet, Copenhagen, Denmark; BRH, Department of Infectious Diseases, Hvidovre Hospital, Copenhagen, Den-mark; CB, Department of Pulmonary and Infectious Diseases, Nordsjællands Hospital Hillerød, Hillerød, Denmark; CØA, Department of Clinical Microbiology, Hvidovre Hospital, Hvidovre, Denmark; HN, Department of Infectious Diseases, Aalborg University Hospital, Aalborg, Denmark.

Table S1

Diagnosis codes used for modified analyses of PPV of ICD-10 codes for brain abscess in the Danish National Patient Registry

ICD-10 diagnosis code	Description
Brain abscess codes
DG060	Intracranial abscess and granuloma
DG060C	Otogenic brain abscess
DG060E	Cerebellar abscess
DG060F	Cerebral abscess
DG079	Intracranial and intraspinal abscess and granuloma in diseases classified elsewhere
DG079B	Intracranial abscess in disease classified elsewhere
DA066	Amebic brain abscess
DB431	Phaeomycotic brain abscess
CNS cancer codes
DC700-DC709	Malignant neoplasms of meninges
DC710-DC719	Malignant neoplasms of brain
DC793	Secondary malignant neoplasm of brain and cerebral meninges
DC794	Secondary malignant neoplasm of other and unspecified parts of nervous system
Spondylodiscitis codes
DM462	Osteomyelitis of vertebra
DM463	Infection of intervertebral disc
DM464	Discitis, unspecified
DM465	Other infective spondylopathies
DG061	Intraspinal abscess and granuloma
Neurosurgical procedure codes
KAAM00	Puncture and aspiration of brain abscess
KAAM10	Excision of brain abscess
KAAM30	Evacuation of epi- or subdural empyema

Abbreviations: CNS, central nervous system; PPV, positive predictive value.