Assessment of the Accuracy of Using ICD-10 Codes to Identify Systemic Sclerosis.

IMPORTANCE: With the increased use of data from electronic medical records for research, it is important to validate in-patient electronic health records/hospital electronic health records for specific diseases identification using International Classification of Diseases, Tenth Revision (ICD-10) codes.
OBJECTIVE: To assess the accuracy of using ICD-10 codes to identify systemic sclerosis (SSc) in the French hospital database. DESIGN SETTING AND PARTICIPANTS: Electronic health record database analysis. The setting of the study's in-patient database was the Toulouse University Hospital, a tertiary referral center (2880 beds) that serves approximately 2.9 million inhabitants. Participants were patients with ICD-10 discharge diagnosis codes of SSc seen at Toulouse University Hospital between January 1, 2010, and December 31, 2017. MAIN OUTCOMES AND MEASURES: The main outcome was the positive predictive value (PPV) of discharge diagnosis codes for identifying SSc. The PPVs were calculated by determining the ratio of the confirmed cases found by medical record review to the total number of cases identified by ICD-10 code.
RESULTS: Of the 2766 hospital stays, 216 patients were identified by an SSc discharge diagnosis code. Two hundred were confirmed as SSc after medical record review. The overall PPV was 93% (95% CI, 88-95%). The PPV for limited cutaneous SSc was 95% (95% CI, 85-98%). CONCLUSIONS AND RELEVANCE: Our results suggest that using ICD-10 codes alone to capture SSc is reliable in The French hospital database.

Methods

We conducted a validation study using the PMSI (Programme de Médicalisation des Systèmes d’Informations) data, the in-patient health database of the French Health Insurance System Database called SNDS (Système national des données de santé, )6 of Toulouse University Hospital (South of France, 4th city by population in the country, 2880 beds in the hospital) and the Toulouse University Hospital Systemic Sclerosis cohort (n=419).

Studies assessing the accuracy of diagnosis coding by medical chart review are authorized by the National Commission of Information Technology and Liberty (CNIL: French Law on Privacy decision No. 89–117). All hospitalized patients in this hospital received written information on the possibility of such studies.

Data Source

In all private and public hospitals in France, data on all hospital stays are recorded in the PMSI since 2007. Data recorded are the hospital identifier, the date of start and of end of stays, the diagnosis codes, the procedures, the costly drugs dispensed, and whether the stays occurred in a special unit (e.g., resuscitation or intensive care).6,7 The diagnoses are encoded using the ICD-108 by the physician in charge of the patient or by specialized technician that generates the codes from medical charts. For this study, we used Toulouse University Hospital PMSI data corresponding to all stays from January 1, 2010, to December 31, 2017, in this hospital. In this institution, discharge diagnoses are coded by a specialized technician according to national standards. The quality of coding is regularly audited.9

The Toulouse University Hospital Systemic Sclerosis cohort (n=419) includes all consecutive SSc hospital stays at Toulouse University Hospital since 2010 who met the ACR/EULAR 2013 classification criteria for SSc.10 In these database, patient information is recorded electronically.

Population

As previously described,11 we extracted all hospital stays with a primary discharge diagnosis ICD-10 code related to SSc: M34.0: “Progressive systemic sclerosis”, M34.1: “CR(E)ST syndrome”, M34.8: “Other forms of systemic sclerosis”, M34.8 + G73.7: “Other forms of systemic sclerosis with lung involvement”, M38.4 + J99.1: “Other forms of systemic sclerosis with myopathy” and M34.9: “Systemic sclerosis, unspecified” from January 1, 2010, to December 31.

Medical Chart Review

All medical records from this electronic query were individually and independently reviewed by two physicians specialized in SSc (S.D.A. and G.P.) to adjudicate the diagnosis. In the event of a discrepancy, a consensus was reached by discussion (it occurred for two patients). After reviewing medical records, cases were categorized as confirmed (true positive) if the ICD-10 code listed in the administrative database correctly identified the disease of interest (i.e., met the ACR/EULAR 2013 classification criteria for SSc).10 If the ICD-10 code did not match, it was noted as miscoded (false positive), and the actual diagnosis was determined.

Measured Outcomes and Statistical Analysis

The positive predictive value (PPV) and 95% confidence interval (CI) of ICD-10 codes were calculated as the ratio of the confirmed cases found by medical record review to the total number of cases identified by ICD-10 code.

For assessing the sensitivity, we identified all patients included in the Toulouse University Hospital Systemic Sclerosis cohort, who met the ACR/EULAR 2013 classification criteria for SSc,10 between 2010 and 2017 and who were admitted at Toulouse University Hospital. We then assessed whether their hospital stays were encoded using an ICD-10 code corresponding to an SSc disease. All analyses were performed using statistical software SAS®, version 9.4.

Results

Of the 2766 hospital stays during the study period, 216 patients had an SSc discharge diagnosis code. Mean (SD) age was 62 (14) years and 163 patients (75.5%) were females. Medical record review confirmed 200 SSc cases (Table 1).

Table 1

Validation of International Classification of Diseases, Tenth Revision (ICD-10) Codes Used to Identify Systemic Sclerosis Cases in the Toulouse University Hospital in-Patient Database (2010–2017)a

Systemic Sclerosis-Related ICD-10 Code	Description	No.b	Confirmedc	PPV, %
M34.0	Progressive systemic sclerosis	35	33	94
M34.1	CR(E)ST syndrome	56	53	95
M34.8	Other forms of systemic sclerosis	60	53	88
-M34.8 + J99.1	-Systemic sclerosis with lung involvement	26	26	100
-M34.8 + G73.7	-Systemic sclerosis with myopathy	4	3	75
M34.9	Systemic sclerosis, unspecified	65	60	92
Total		216	200	93

Notes: aAll Systemic Sclerosis cases came from codes in this table. bIdentified through electronic search. cConfirmed through medical record review.

Abbreviations: PPV, positive predictive value; CR(E)ST syndrome, combination of calcinosis, Raynaud phenomenon, (o)esophageal dysfunction, sclerodactyly, telangiectasia.

The overall PPV of SSc-specific ICD-10 codes to identify patients with SSc was 93% (95% CI, 88–95%). The PPVs of individual SSc codes are listed in Table 1 and ranged from 75% to 100%. Among them, the PPV of CREST syndrome codes to identify patients with CREST syndrome was 95% (95% CI, 85–98%).

There were 16 cases with SSc-specific ICD-10 codes who were miscoded (Table 2). The most common reasons for excluding a case after medical record review were for a different connective tissue disease (50% [8 of 16]) or localized scleroderma (18.8% [3 of 16]).

Table 2

Reasons for Excluding Patients Identified by Systemic Sclerosis-Specific International Classification of Diseases, Tenth Revision (ICD-10) Codes in the French Hospital Database (2010–2017)a

Reason	No.	Associated Systemic Sclerosis ICD-10 Codesb
Localized Scleroderma	3	M34.0, M34.8, M34.9
Primitive Sjogren’s syndrome	3	M34.8, M34.9b
Undifferentiated Connective Tissue Diseases	3	M34.9b
Scleredema Adultorum	2	M34.8, M34.9
Raynaud Disease	1	M34.8
Systemic Lupus Erythematosus	1	M34.1
Dermatomyositis	1	M34.8
Multiple Sclerosis	1	M34.8
HCV-related cryoglobulinemia vasculitis	1	M34.8
Total excluded	16

Notes: aThere were 216 patients with a systemic sclerosis-specific code. bCodes contributing more than 1 case.

Abbreviation: HCV, hepatitis C virus.

The ICD-10 discharge diagnosis code M34.9 (Systemic sclerosis, unspecified) contributed the most confirmed SSc cases (30% [60 of 200]). The ICD-10 discharge diagnosis code M34.8 (Other forms of systemic sclerosis) had the most false-positive results: 43.8% (7 of 16) of the excluded cases were from code M34.8.

During the same period, 172 SSc patients were included in the Toulouse University Hospital SSc cohort following a hospitalization. The initial hospital stay was encoded M34.0, M34.1, M34.8, or M34.9 for 156 out of these patients. Consequently, the sensitivity of the SSc discharge diagnosis codes was 91% (95% CI, 85–94%). The 16 false-negative cases were all coded L94.0 (localized scleroderma).

Discussion

To the best of our knowledge, this is the first study to assess the accuracy of SSc-specific ICD-10 SSc discharge diagnosis codes in an electronic administrative database. We found the accuracy of electronic medical records for identifying patients with SSc to be excellent in the French hospital database.

While billing code accuracy studies have been performed for other connective tissue diseases,12,13 they have been lacking for SSc. One published14 study reported that the accuracy of ICD-9 codes for SSc (710.1 code) was 76%. In this study, the true positive cases were defined by the fulfillment of at least one of the three different classification criteria, including the updated 2013 ACR/EULAR criteria.10 ICD-9 only has one code for SSc while ICD-10 has six, this difference could be an explanation why the accuracy may now be higher.

Our study suggests that most of the false-negative cases were coded “localized scleroderma” this suggests that in order to improve the sensitivity of SSc discharge diagnosis code technicians may need further training to better differentiate SSc from localized scleroderma.

Our findings indicate that, given the strong agreement between the ICD-10 codes and medical records, SSc can confidently be studied using administrative claims data. Our study shows that PPVs of ICD-10 codes for SSc are comparable to PPVs of common conditions,15–17 including cataract, diabetes mellitus, acute myocardial infarction, or stroke (PPV >90% for all). Our study shows that PPVs of ICD-10 codes for SSc are more reliable than PPVs of other connective tissue diseases,18,19 such as systemic lupus erythematous and dermatomyositis (PPV <90% for all).

Our Study Has Several Limitations to Consider

It is limited to a single tertiary center, not necessarily reflecting PMSI coding in the entire France. Even if coding rules are national, there may be differences between establishments. The generalizability of this study may be limited because there may be potential differences in coding in a contained healthcare system like Toulouse University Hospital compared with other systems where insurance approval is needed for medications, procedures, and treatments. Future studies will need to assess the accuracy of ICD-10 codes for SSc in other settings.

Conclusion

In summary, this study provides information on the accuracy of ICD-10 discharge diagnostic codes for SSc in the French hospital database. Consequently, this database is a useful tool for assessing the epidemiology and pharmacoepidemiology of SSc. Given the accuracy of SSc codes, claims data appear to be sufficient for identifying SSc patients for electronic medical record studies. These investigations can be valuable, but the intricacies of each disease and database should be carefully considered when designing studies.

Article Summary: Strengths and Limitations of This Study

An electronic health record database analysis showed that the positive predictive value of SSc discharge diagnosis codes was 93%.

The sensitivity of the SSc discharge diagnosis codes was 91%.

These data suggest that ICD-10 discharge diagnosis codes can be used to identify cases of systemic sclerosis in the French hospital database.

Systemic sclerosis (SSc) is a systemic orphan connective tissue disease which accounts for significant morbi-mortality worldwide.1 Despite the increasing number of population-based reports using health databases for research in connective tissue diseases,2 few studies have assessed the accuracy of International Classification of Diseases, Tenth Revision (ICD-10) diagnostic codes for SSc.3,4 Because claims database are primarily used for billing purposes, it is important to validate the use of diagnostic codes for research and ensure that the coding actually reflects the conditions described in medical records.5

In this study, we assessed the accuracy of ICD-10 discharge diagnosis codes to identify SSc, in the French Hospital Database, compared with medical record review as the gold standard using the Toulouse University Hospital database, a tertiary medical center. In this database, patient information is recorded electronically. These findings will help guide future studies aimed at using electronic medical records to perform research on SSc.