Positive predictive values of International Classification of Diseases, 10th revision codes for dermatologic events and hypersensitivity leading to hospitalization or emergency room visit among women with postmenopausal osteoporosis in the Danish and Swedish national patient registries.

BACKGROUND: Clinical epidemiology research studies, including pharmacoepidemiology and pharmacovigilance studies, use routinely collected health data, such as diagnoses recorded in national health and administrative registries, to assess clinical effectiveness and safety of treatments. We estimated positive predictive values (PPVs) of International Classification of Diseases, 10th revision (ICD-10) codes for primary diagnoses of dermatologic events and hypersensitivity recorded at hospitalization or emergency room visit in the national patient registries of Denmark and Sweden among women with postmenopausal osteoporosis (PMO).
METHODS: This validation study included women with PMO identified from the Danish and Swedish national patient registries (2005-2014). Medical charts of the potential cases served as the gold standard for the diagnosis confirmation and were reviewed and adjudicated by physicians.
RESULTS: We obtained and reviewed 189 of 221 sampled medical records (86%). The overall PPV was 92.4% (95% confidence interval [CI], 85.1%-96.3%) for dermatologic events, while the PPVs for bullous events and erythematous dermatologic events were 52.5% (95% CI, 37.5%-67.1%) and 12.5% (95% CI, 2.2%-47.1%), respectively. The PPV was 59.0% (95% CI, 48.3%-69.0%) for hypersensitivity; however, the PPV of hypersensitivity increased to 100.0% (95% CI, 67.6%-100.0%) when restricting to diagnostic codes for anaphylaxis. The overall results did not vary by country.
CONCLUSION: Among women with PMO, the PPV for any dermatologic event recorded as the primary diagnosis at hospitalization or at an emergency room visit was high and acceptable for epidemiologic research in the Danish and Swedish national patient registries. The PPV was substantially lower for hypersensitivity leading to hospitalization or emergency room visit.

Introduction

Adverse drug reactions such as dermatologic events and hypersensitivity are common health conditions, which account for 3%–6% of all hospital admissions and occur in 10%–15% of hospitalized patients.1 Adverse drug reactions are side effects of many medical treatments for chronic diseases.

Postmarketing monitoring of drug safety using health and administrative registries and/or databases is essential.2 Therefore, assessing the validity of diagnoses for adverse events in the registries is important to ensure high quality of data used for pharmacovigilance studies. National health registries in Nordic countries have long been recognized as well suited for pharmacoepidemiologic research3 and have increasingly relied on study drug safety, especially in relation to treatments of chronic conditions for which drugs are dispensed through outpatient pharmacies.

However, the validity of diagnoses of common drug reactions such as dermatologic events and hypersensitivity found in the Nordic patient registries is not known.4,5 In this study, we estimated positive predictive values (PPVs) of case ascertainment algorithms for dermatologic events and hypersensitivity in the national hospital-based patient registries of Denmark and Sweden among women with postmenopausal osteoporosis (PMO), a disease where several new drugs have been introduced in the past 2 decades.

Methods

Setting

We conducted a population-based validation study in Denmark and Sweden between January 1, 2005, and December 31, 2014, in a setting of universal health care access and routine recording of health events.6 The source population consisted of postmenopausal women (age 55 years or older). The Danish source population was restricted to the Central Denmark Region (size of the postmenopausal women population on January 1, 2011, was 189,319).7 Aarhus University Hospital is the largest hospital serving the area. In Sweden, the source population comprised postmenopausal women residing in areas served by the 5 largest Stockholm-area hospitals, Karolinska University Hospital, Solna and Huddinge, Danderyd Hospital, The South Hospital, and St Göran Hospital. The size of the postmenopausal women population residing in the Stockholm County on December 31, 2010, was 290,179.8

Study population

The study population included potential cases of dermatologic events and hypersensitivity among women with PMO seen in the hospitals in the selected geographic areas of Denmark and Sweden. The definition of PMO was based on an algorithm that included diagnostic codes indicating osteoporosis diagnosis, diagnosis of osteoporotic (fragility) fracture, or use of osteoporosis medications, among women 55 years of age or older. This cohort was initially assembled for an ongoing pharmacovigilance study, which has been described in detail elsewhere.9 To be included in the cohort, a woman had to meet at least 1 component of the osteoporosis algorithm on or after her 55th birthday (Table S1). The index date was the date on which a woman first satisfied the inclusion criteria. Women with a diagnosis of Paget’s disease or a diagnosis of any malignancy (except nonmelanoma skin cancer) during the previous 12 months before the index date were excluded (Table S2). Potential cases were identified from the Danish National Patient Registry10 and the Swedish Patient Register11 using the International Classification of Diseases, 10th revision (ICD-10) diagnostic codes. Potential cases were defined as ICD-10 codes indicative of hypersensitivity or a dermatologic event recorded as the primary diagnosis during a hospitalization or an emergency room/unplanned visit. The ICD-10 codes used to identify the potential cases are listed in Tables S3 and S4. A priori, we planned to sample 100 potential cases of dermatologic events (50 for Denmark and 50 for Sweden) and 100 potential cases of hypersensitivity (50 for Denmark and 50 for Sweden). In Denmark, we included all patients with dermatologic events (N=42) and hypersensitivity (N=32) from the period 2005–2012 and from Aarhus University Hospital sited in the Central Region of Denmark. In addition, we randomly sampled 8 and 18 patients, respectively, from other hospitals in the Central Region in order to achieve a sample pool of 50 patients for each of the events. However, there were limitations on access to the medical records necessary for the review, and hence we decided to increase the sample pools to 60 by samples of patients treated at Aarhus University hospital in 2013 and 2014. The number of selected potential cases remained 50 each for dermatologic events and hypersensitivity in Sweden.

Medical record review

The sampled potential cases were confirmed through review of medical records (paper or electronic). We attempted to obtain all available medical records, which were reviewed and adjudicated by physicians. The adjudicating physicians confirmed the case status in 3 categories, according to predefined clinical criteria: 1) definite case, 2) definite noncase, or 3) insufficient information. In addition, we verified that each event (dermatologic reaction and hypersensitivity) was the primary diagnosis that led to a hospitalization or an emergency/unplanned visit. If neither definite case nor definite noncase status could be assigned by adjudicators, the case was categorized as having insufficient information and excluded from the analysis. Screen shots of the medical extraction forms are given in Figures S1 and S2.

Statistical analyses

From the available registry data, we compiled descriptive data on patients’ age, country, case year, department, and type of hospital visit (planned vs unplanned [emergency]). Unplanned hospital visits served as a proxy for emergency room visits as specific codes for the latter are lacking in the Swedish Patient Registers. The PPVs were calculated as the proportion of potential cases that could be classified as definite cases by medical chart review. Because bullous or erythematous conditions as well as anaphylactic hypersensitivity are more severe conditions, which are frequently of interest in pharmacoepidemiology studies on drug safety, we conducted additional analyses on these specific subtypes. Moreover, we restricted the PPV calculation of hypersensitivity to codes for anaphylaxis. All PPVs were reported with 95% confidence intervals (CIs) calculated according to the Wilson score interval method.12 Country-specific stratified analyses were conducted separately for dermatologic events and hypersensitivity leading to hospitalization or emergency room visit. Patient sampling and statistical analyses were performed using SAS version 9.2, SAS Institute Inc., Cary, NC, USA.

Ethical considerations

In Denmark, the study was approved by the Danish Data Protection Agency (record number 2010-41-5171) and by the Data Protection Board of the Danish Central Region (record number 1-16-02-1-08). In Sweden, the Stockholm County Regional Ethics Review Board approved the study (record number 2010/1617-31/3). According to Danish and Swedish law, informed consent from patients is not required for registry-based studies.

Results

Overall, 221 potential cases were identified; of these, 32 patients had missing information in the medical record or the medical record file could not be found (Figure 1). Medical records were available for 189 patients (86%), including 100 potential cases of dermatologic events (mean age 77 years) and 89 potential cases for hypersensitivity (mean age 76 years; Tables 1 and 2). Of these, 8 potential cases (8%) with dermatologic events and 6 potential cases with hypersensitivity (7%) had insufficient information for medical adjudication and were not included in the PPV calculations (Table 3).

Figure 1

Flow chart showing selection of potential cases of dermatologic events and hypersensitivity in the national patient registries in Denmark and Sweden.

Table 1

Characteristics of patients with dermatologic events leading to hospitalization or emergency room visit among women with postmenopausal osteoporosis

	Requested, N = 110	Obtained, N = 100
Age (mean, SD)	76 (10.8)	77 (10.8)
Country (n, %)
Denmark	60 (54.5)	50 (50.0)
Sweden	50 (45.5)	50 (50.0)
Case year (n, %)
2005	10 (9.1)	7 (7.0)
2006	16 (14.5)	15 (15.0)
2007	17 (15.5)	17 (17.0)
2008	18 (16.4)	17 (17.0)
2009	13 (11.8)	11 (11.0)
2010	11 (10.0)	10 (10.0)
2011	6 (5.5)	4 (4.0)
2012	9 (8.2)	9 (9.0)
2013	2 (1.8)	2 (2.0)
2014	8 (7.3)	8 (8.0)
Hospital department (n, %)
Internal medicinea	39 (35.4)	38 (38.0)
Dermatology	51 (46.4)	45 (45.0)
Emergency room	16 (14.5)	16 (16.0)
Surgery	4 (3.6)	1 (1.0)
Type of hospital visit (n, %)
Planned	72 (65.5)	65 (65.0)
Unplanned	38 (34.5)	35 (35.0)

Notes:

Covering departments of cardiology, endocrinology, infectious disease, and general internal medicine. All covariates are assessed on the case date (age, case year, provider type) or during the 12-month look back period before the case date. Case date is the date the potential case was identified.

Abbreviation: SD, standard deviation.

Table 2

Characteristics of patients with hypersensitivity leading to hospitalization or emergency room visit among women with postmenopausal osteoporosis

	Requested, N = 111	Obtained, N = 89
Age (mean, SD)	76 (10.6)	76 (10.6)
Country (n, %)
Denmark	60 (54.1)	38 (42.7)
Sweden	51 (45.9)	51 (57.3)
Case year (n, %)
2005	6 (5.4)	5 (5.6)
2006	10 (9.0)	8 (9.0)
2007	5 (4.5)	5 (5.6)
2008	23 (20.7)	19 (21.3)
2009	21 (18.9)	18 (20.2)
2010	9 (8.1)	4 (4.5)
2011	13 (11.7)	9 (10.1)
2012	14 (12.6)	11 (12.4)
2013	4 (3.6)	4 (4.5)
2014	6 (5.4)	6 (6.7)
Hospital department (n, %)
Internal medicinea	79 (71.8)	73 (82.0)
Emergency room	12 (10.8)	10 (11.2)
Geriatrics	4 (3.6)	4 (4.5)
Surgery	15 (13.5)	2 (2.2)
Others	1 (0.9)	0 (0.0)
Type of hospital visit (n, %)
Planned	65 (58.6)	58 (65.2)
Unplanned	46 (41.4)	31 (34.8)

Notes:

Covering departments of cardiology, endocrinology, infectious disease, general internal medicine, and pulmonology. All covariates are assessed on the case date (age, case year, provider type) or during the 12-month look back period before the case date. Case date is the date the potential case was identified.

Abbreviation: SD, standard deviation.

Table 3

Summary of medical review PPVs for dermatologic events and hypersensitivity among women with postmenopausal osteoporosis in Denmark and Sweden

Category	Obtained medical charts (n)	Potential cases with insufficient information, n (%)	Confirmed casesa after medical chart review (n)	PPV (95% CI)b
Dermatologic events leading to hospitalization or ER visit	100	8 (8.0)	85	92.4 (85.1–96.3)
Dermatologic subtype: bullous dermatoses	40	0 (0.0)	21	52.5 (37.5–67.1)
Dermatologic subtype: erythematous	8	0 (0.0)	1	12.5 (2.2–47.1)
Hypersensitivity leading to hospitalization or ER visit	89	6 (6.7)	49	59.0 (48.3–69.0)
Hypersensitivity subtype: anaphylactic hypersensitivity	8	0 (0.0)	8	100.0 (67.6–100.0)

Notes:

Confirmed cases based on medical chart review.

PPV is calculated as the number of confirmed cases after medical review divided by the number of obtained medical charts with sufficient information.

Abbreviations: CI, confidence interval; ER, emergency room; PPV, positive predictive value.

Information verifying a dermatologic event leading to hospitalization or emergency room visit was found in 85 of 92 medical records, yielding a PPV of 92.4% (95% CI, 85.1%–96.3%; Table 3). The PPV was substantially lower for bullous dermatoses (52.5% [95% CI 37.5%–67.1]) as well as for erythematous pathological subtype (12.5% [95% CI 2.2%–47.1%]; Table 3). Hypersensitivity leading to hospitalization or emergency room visits was confirmed in 49 of 83 medical records, yielding a PPV of 59.0% (95% CI: 48.3%–69.0%). The PPV was 100.0% (95% CI, 67.6%–100%) for hypersensitivity when restricting PPV calculation to codes for anaphylaxis.

The overall PPVs of dermatologic events and hypersensitivity were comparable in Denmark and Sweden (Table 4).

Table 4

PPVs for dermatologic events and hypersensitivity among women with postmenopausal osteoporosis stratified by country

Category	Obtained medical charts (n)	Potential cases with insufficient information, n (%)	Confirmed cases after medical chart review (n)a	PPV (95% CI)b
Denmark
Dermatologic events leading to hospitalization or ER visit	50	0 (0.0)	46	92.0 (81.2–96.8)
Dermatologic subtype: bullous dermatoses	30	0 (0.0)	21	70.0 (52.1–83.3)
Dermatologic subtype: erythematous	4	0 (0.0)	1	25.0 (4.6–69.9)
Hypersensitivity leading to hospitalization or ER visit	38	3 (7.9)	20	57.1 (40.9–72.0)
Hypersensitivity subtype: anaphylactic hypersensitivity	3	0 (0.0)	3	100.0 (43.9–100.0)
Sweden
Dermatologic events leading to hospitalization or ER visit	50	8 (16.0)	39	92.9 (81.0–97.5)
Dermatologic subtype: bullous dermatoses	10	0 (0.0)	0	0.0 (0.0–27.8)
Dermatologic subtype: erythematous	4	0 (0.0)	0	0.0 (0.0–49.0)
Hypersensitivity leading to hospitalization or ER visit	51	3 (5.9)	29	60.4 (46.3–73.0)
Hypersensitivity subtype: anaphylactic hypersensitivity	5	0 (0.0)	5	100.0 (56.6–100.0)

Notes:

Confirmed cases based on medical chart review.

PPV is calculated as the number of confirmed cases after medical review divided by the number of obtained medical charts with sufficient information.

Abbreviations: CI, confidence interval; ER, emergency room; PPV, positive predictive value.

Discussion

Among women with PMO in Denmark and Sweden, we adjudicated hospital-based primary diagnoses of dermatologic events and hypersensitivity leading to hospitalization or emergency room visit coded in national patient registries. The PPV was >90% for dermatologic events, but substantially lower for hypersensitivity. The higher PPV for dermatologic events than for hypersensitivity may reflect differences in the complexity of the disease definition and that ∼50% of the potential dermatologic event cases originated from specialist departments of dermatology, whereas >90% of the potential hypersensitivity cases originated from departments of internal medicine and emergency rooms. The term hypersensitivity was often found to be used for normal physiological reactions of overdoses, for example, a high international normalized ratio (INR) as a result of too much warfarin. Most likely this mirrors the lack of time to find a more correct code.

To the best of our knowledge, our study is the first to estimate the PPV of ICD-10 codes for identifying patients with dermatologic events and hypersensitivity in the Danish and Swedish national patient registries. Only 2 studies have assessed the validity of ICD-10 codes for skin diseases in the Danish National Patient Registry with medical records as reference. A study of 38 cancer patients estimated a PPV almost as high as ours, 79% (95% CI, 64%–90%) for skin infections when using an abstraction form and physician assessment in the same manner as done in our study, but the PPV was lower when confirmed by evidence-based criteria (45%, 95% CI, 30%–60%), requiring, for example, at least 7 days of antibiotics.5 Another study of 589 stroke patients validated several medical complications, among them the diagnosis of decubitus using a standardized abstraction form, but without physician assessment. With only 8 cases in the Danish National Patient Registry, where 4 were verified, the PPV was found to be 50% (95% CI, 16%–84%) for decubitus.4 Skin disease codes in the Swedish National Patient Registry have only been evaluated in 1 previous study.13 Among children younger than 17 years, who were prescribed glucocorticoid or immunosuppressant medication recorded in the Swedish Prescribed Drug Register, the PPVs were 82% for unspecific dermatitis (n=199) and 45% for the eczema (n=108), with medical record review conducted by physicians as reference standard.13

In 2012, the American Food and Drug Administration (FDA) conducted a systematic review of US studies validating algorithms for anaphylaxis and related conditions. They concluded that more research is needed; more validation studies to test anaphylaxis algorithms need to be conducted. They found only 6 studies, with varying PPVs (38%–72%) depending on cohorts and ICD codes used.14 Taken together, the literature in the area is limited and methods to validate differ. Previous validation studies have reported substantial variation in PPVs. This was also seen in our study of dermatologic events and hypersensitivity.

The Danish and Swedish national patient registries offer a variety of possibilities for pharmacoepidemiologic studies.10 Based on our findings, the national patient registries can be used to study cohorts of patients with dermatologic events as well as postmarketing monitoring of dermatologic events and hypersensitivity of treatments. However, the low PPV of our algorithm for identifying hypersensitivity leading to hospitalization or emergency room visits is not adequate for monitoring in pharmacovigilance studies if absolute risk is the outcome of interest. In comparative safety studies, however, relative risks should be unbiased if specificity of the diagnosis is high and misclassification is nondifferential across exposure categories.15

Strengths and limitations

This study was conducted among women with PMO in selected areas and hospitals of Denmark and Sweden. Scandinavian countries are welfare states with universal income-independent access to health care and uniform health care delivery. Hence, results of this validation study are likely generalizable to the overall postmenopausal female population in each country, although variation across geography and hospital size cannot be ruled out. By design, this study assessed PPVs and could not assess sensitivity because of a lack of an independent sample of confirmed cases.

Conclusion

Our study showed that primary diagnoses of dermatologic events recorded at hospitalization or emergency room/unplanned visit have high PPVs in the Danish and Swedish national patient registries. In contrast, the PPV for hypersensitivity leading to hospitalization or emergency room visit was substantially lower. Thus, depending on study aims, the Danish and Swedish national patient registries may be useful for studying dermatologic events, but may not be adequate for studying hypersensitivity events.