Validation of an algorithm for identifying MS cases in administrative health claims datasets.

OBJECTIVE: To develop a valid algorithm for identifying multiple sclerosis (MS) cases in administrative health claims (AHC) datasets.
METHODS: We used 4 AHC datasets from the Veterans Administration (VA), Kaiser Permanente Southern California (KPSC), Manitoba (Canada), and Saskatchewan (Canada). In the VA, KPSC, and Manitoba, we tested the performance of candidate algorithms based on inpatient, outpatient, and disease-modifying therapy (DMT) claims compared to medical records review using sensitivity, specificity, positive and negative predictive values, and interrater reliability (Youden J statistic) both overall and stratified by sex and age. In Saskatchewan, we tested the algorithms in a cohort randomly selected from the general population.
RESULTS: The preferred algorithm required ≥3 MS-related claims from any combination of inpatient, outpatient, or DMT claims within a 1-year time period; a 2-year time period provided little gain in performance. Algorithms including DMT claims performed better than those that did not. Sensitivity (86.6%-96.0%), specificity (66.7%-99.0%), positive predictive value (95.4%-99.0%), and interrater reliability (Youden J = 0.60-0.92) were generally stable across datasets and across strata. Some variation in performance in the stratified analyses was observed but largely reflected changes in the composition of the strata. In Saskatchewan, the preferred algorithm had a sensitivity of 96%, specificity of 99%, positive predictive value of 99%, and negative predictive value of 96%.
CONCLUSIONS: The performance of each algorithm was remarkably consistent across datasets. The preferred algorithm required ≥3 MS-related claims from any combination of inpatient, outpatient, or DMT use within 1 year. We recommend this algorithm as the standard AHC case definition for MS.

The United States has historically been considered a region with a high prevalence of multiple sclerosis (MS). However, the prevalence of MS remains poorly described,[1] and rigorous national estimates are lacking. Prevalence estimates are critical for understanding the national burden of MS, planning for health service needs, and supporting advocacy efforts.

Multiple methods have been used to estimate the prevalence of MS worldwide, including surveys, review of medical records or lists from health care institutions, and administrative health claims (AHC) data.[1-3] AHC data are attractive because of their relatively low cost, ease of use relative to other methods, and potential for repeated use over time. However, AHC data are not collected for research purposes, and their validity must be assessed before use.[4-6] Validation studies are possible for some AHC datasets in the United States such as the Department of Veterans Affairs (VA) but not in others such as Medicare because they cannot be linked back to clinical records.

Therefore, we aimed to test several case definitions (algorithms) for identifying persons with MS using multiple AHC datasets. We reasoned that if we could identify a best-case definition that performed well and comparably across diverse AHC datasets, this would support their use in other health claims datasets in which validation studies were not feasible. Further details and rationale for this approach are provided in one of our companion articles (Nelson et al.,[7] “A New Way to Estimate Neurologic Disease Prevalence in the United States: Multiple Sclerosis”). Second, we aimed to determine how much the duration of the observation period would affect case detection (i.e., underestimate prevalence) for appropriate adjustment in our companion prevalence article (Wallin et al.,[8] “The Prevalence of MS in the United States: A Population-Based Estimate Using Health Care Claims Data”).

Methods

This was a retrospective analysis using linked administrative and clinical data that was conducted in parallel at 3 primary sites (dataset characteristics in table e-1 available from Dryad, doi.org/10.5061/dryad.4c7s325) following a common protocol. The findings from the initial analysis were then tested in a fourth independent dataset. Sources of the AHC datasets are described below. Clinical data were drawn from medical records. Neurologist-confirmed diagnoses of MS based on the 2010 McDonald criteria[9] served as the reference standard for this study. At the 3 primary sites, the source population for this study was limited to persons ≥18 years of age who were alive during the study period (2000–2014) and had ≥1 health care encounter associated with a diagnostic code for MS (Manitoba used any demyelinating disease to define its source population).

Standard protocol approvals, registrations, and patient consents

Each site obtained local Institutional Review Board or ethics approval: University of Maryland, Baltimore; Baltimore VA Research and Development Committee; Kaiser Permanente Southern California (KPSC); University of Manitoba Health Research Ethics Board; and University of Saskatchewan Biomedical Research Ethics Board. In Manitoba, approval for administrative data access was obtained from the Health Information Privacy Committee. In Saskatchewan, approval for data access was obtained from the Saskatchewan Ministry of Health and the Saskatchewan Health Quality Council.

Data sources

Department of Veterans Affairs

The VA provides comprehensive health care to ≈11 million military veterans at ≈300 hospitals and 1,500 outpatient clinics and Veterans Centers distributed throughout the United States, Puerto Rico, and Guam. Since 1997, the VA has prospectively collected and maintained extensive data on all funded health care services, including hospitalizations, physician visits, and prescriptions dispensed. Diagnoses were recorded using ICD-9 codes.

The initial validation cohort was drawn from the VA case-finding algorithm study in 2006,[5] which included all patients in the mid-Atlantic region who were ≥18 years of age with ≥1 encounter with a diagnosis code for MS (ICD-9 code 340). We added data from the Gulf War Era MS Cohort[10] and data from screening of patients for a planned longitudinal MS study. We followed the same standardized medical record review procedures (trained abstractors under the direction of M.W.) to verify the diagnosis of MS[9] for all patients. In total, these studies yielded a cohort of 3,452 patients (table 1) from 1999 through 2007.

Table 1

Demographic characteristics of the MS validation cohorts by data source

Kaiser Permanente Southern California

KPSC is a prepaid health maintenance organization with a membership during the study period of >3.5 million persons and provides health care coverage to ≈20% of the population in the region it serves. The KPSC membership is largely representative of the general population in Southern California with respect to ethnicity, age, and sex but underrepresents the lowest and highest ends of the socioeconomic spectrum.[11] Physician visits, hospitalizations, and medications are captured in datasets that are linkable by a medical record number.

The validation cohort consisted of all persons ≥18 years of age with ≥1 inpatient (IP) or outpatient (OP) encounter diagnosis code for MS (ICD-9 code 340) who were KPSC health plan members between 2008 and 2010, had at least 6 months of continuous membership during the study period, and had complete electronic health records (n = 4,851). We randomly selected 2,935 (table 1) of these potential cases for in-depth medical records review by an MS specialist (A.L.-G), including IP and OP records, MRI scans, and diagnostic test results, to confirm a diagnosis of MS.[9]

Manitoba

In the Canadian province of Manitoba, Manitoba Health, Seniors and Active Living is responsible for the delivery of publicly funded health care to nearly all residents. Several datasets are maintained that capture demographic information (sex, dates of birth, health care coverage, and death), hospitalizations, physician services, and prescription drug dispensations. Diagnoses for physician services are recorded with the clinical modification of ICD-9 (ICD-9-CM) codes, while diagnoses for hospitalizations are coded with ICD-9-CM or the Canadian version of ICD-10 codes (ICD-10-CA), depending on the year.

The Manitoba validation cohort started with the 400 patients ≥18 years of age who were used to develop administrative case definitions of MS in 2007.[6] This cohort was augmented by review of the medical records of all persons participating in the MS Clinic Registry (trained abstractors under the supervision of R.A.M.) at the Winnipeg MS Clinic who had consented to linkage of their medical records to administrative data (89% of those approached) as of June 2015 and met the inclusion criteria as described above. This expanded the validation sample to 1,654 patients (table 1).

Saskatchewan

This constituted the fourth independent site. In the Canadian province of Saskatchewan, health care is publicly funded as in Manitoba. Datasets similar to those in Manitoba capture demographic information (sex, dates of birth, health care coverage, and death), hospitalizations, physician services, and prescription drug dispensations. Diagnoses for physician services are recorded with ICD-9-CM codes, while diagnoses for hospitalizations are coded with ICD-9 or ICD-10-CA codes, depending on the year.

As described elsewhere,[12] the validation cohort included 200 randomly selected cases from the Saskatoon MS Clinic with diagnoses confirmed by an MS specialist and 200 controls (table 1) from the Inpatient Rehabilitation Center database. This database specifically captures diagnoses of chronic diseases, including MS.[13]

MS case definition algorithms

We developed candidate algorithms using ICD-9/ICD-10 codes for MS (340/G35) and lists of MS-specific disease-modifying therapies (DMTs). To develop these definitions, we reviewed the literature on validated administrative case definitions for MS.[5,6,12-15] From this review, we considered the following: (1) 1 encounter with an ICD-9/ICD-10 code for MS is too nonspecific because these may represent rule-out codes; (2) increasing the number of health care encounters required generally increases specificity at the expense of sensitivity; and (3) prescription claims for MS-specific DMTs may be helpful to improve sensitivity, specificity, or both when available. Therefore, the candidate algorithms (table 2) varied the number of IP and OP encounters with MS listed as one of the diagnoses for that encounter, the number of prescription claims for a DMT required to classify a person as an MS case, and the time period over which the case definition should be met. We included candidate algorithms that did not use prescription claims because these are not available in all AHC datasets. Because we anticipated applying these definitions in AHC datasets in which enrollment in a health insurance plan might be limited to short time periods, we considered only 1- or 2-year time periods over which to define a case. Each component of these case definitions is described further below.

Table 2

Description of candidate algorithms for MSa

IP encounter

We defined an IP encounter as an IP admission (overnight stay) for which MS was recorded as one of the diagnoses. To account for transfers between institutions and to avoid double counting, we collapsed multiple overlapping hospital admission and discharge records into 1 IP encounter. In addition, an IP encounter with an admission date that occurred within 24 hours of a prior IP encounter was counted as a single IP encounter.[5,6]

Physician/OP encounters

We defined OP encounters as a visit to an OP clinic with an ICD code for MS. Multiple OP visits by 1 patient within the same day were treated as 1 OP encounter. In addition, OP encounters occurring within an MS IP (admission to discharge date) were not counted as unique OP encounters.

Disease-modifying therapies

We defined prescription claims as dispensations for any of the MS-specific DMTs approved before or during the study period, including interferon beta-1a-SC, interferon beta-1a-IM, interferon beta-1b-SC, glatiramer acetate, fingolimod, and natalizumab as identified with National Drug Codes and Drug Identification numbers (tables e-1 and e-2 available from Dryad, doi.org/10.5061/dryad.4c7s325). To avoid misclassification, claims for natalizumab were not included if the individual also had an ICD code for inflammatory bowel disease, a disorder for which this medication is approved for use.

A caveat in defining IP and OP encounters was that the VA imposes a hierarchy to the coding such that the primary diagnosis for a given encounter should be listed first, followed, in descending order of importance, by other diagnoses affecting that episode of care. Not all AHC datasets impose such a hierarchy (e.g., KPSC). Our assessment of the effect of this coding hierarchy (see Appendix 2 available from Dryad for additional methods and results, doi.org/10.5061/dryad.4c7s325) resulted in using MS as a primary diagnosis in the VA data and MS as any diagnosis in the KPSC and Manitoba datasets when counting MS encounters.

Analysis

We applied the candidate algorithms in each of the 3 primary validation cohorts. We compared the performance of the candidate algorithms with the reference standard diagnoses of MS from medical records using sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). In addition, interrater agreement was compared between the algorithms derived from AHC and medical records data using the crude accuracy score and Youden[16] J statistic. We chose the J statistic over the κ statistic[17,18] because it is designed for use with dichotomous data and equally weights false-positives and false-negatives. We interpreted the J statistic like κ, when neither the algorithm nor the medical records were considered the reference standard, as slight (0–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), and excellent (0.81–1.0).[19]

To determine whether the algorithms performed consistently across sociodemographic subgroups, we stratified the reference population by (1) sex, and (2) age group (18–24, 25–34, 35–44, 45–54, 55–64, 65–74, ≥75 years) and repeated our analyses for the 4 algorithms that performed best across the 3 AHC datasets. The ultimate choice of an algorithm was based on performance and pragmatic considerations, including ease of application and interpretation.

Because our 3 primary validation datasets were drawn from patients with at least 1 encounter for MS (or demyelinating disease in Manitoba), not the general population, we expected that the specificity and predictive values of the case definitions would be artificially depressed compared to the performance in the general population. Therefore, we tested our case definition algorithms in a fourth independent dataset in Saskatchewan,[14] which included individuals with and without MS as noted above.

Prevalence with limited duration ascertainment periods

We intended to apply our optimal algorithm to multiple AHC datasets available in the United States[20] to determine prevalence in 2010 based on a 3-year ascertainment period. However, limited periods of observation may substantially reduce case detection and underestimate prevalence, as is well recognized in the literature.[21-23] Capocaccia et al.[22] describe how they dealt with adjusting data from cancer registries that had been active for short durations when creating the Europe-wide project to estimate the prevalence of the most important cancers (EUROPREVAL). They computed what they called a completeness index for multiple observation periods using data from the Connecticut Cancer Registry because it has been in operation for >50 years. In the 4 registries that had only 5 observation years, the completeness index ranged from 0.42 to 0.45. The completeness index improved systematically as the observation period increased. They go on to describe their model-based methods to estimate the complete prevalence based on a limited period using age- and sex-specific incidence and survival rates for cancer.[22] This information is lacking in the US MS population and cannot be accurately estimated from data of only 3 years’ duration.[23] Therefore, to assess the effect of a short ascertainment period, we used the approaches used by Ng et al.[21] and Capocaccia et al.[22] and estimated the prevalence of MS using 3 years (2008–2010) and 10 years (1999–2010) of data in the VA and Manitoba datasets and compared the difference in these estimates in 2010. Prevalence in each year was defined as all persons ≥18 years meeting our preferred algorithm definition of MS ([IP + OP + DMT] ≥ 3) among those alive during the calendar year. The 3-year and 10-year prevalence estimates in 2010 were cumulative estimates based on all those who had met the definition at any point in the ascertainment period and who were alive in 2010.

To validate the above analyses, we repeated these procedures using the PharMetrics Plus Health Plan Claims Dataset from Intercontinental Marketing Services (IMS), which was not used to develop the algorithm. Here, we compared the estimated prevalence derived from 3 years (2013–2015) and 9 years (the longest period available) ending in 2015. The IMS dataset captures health care claims from ≈120 health plans across the United States accounting for ≥42 million covered lives as of 2011, mirrors the demographics of the US Census population, and has been used for national and regional benchmarking of health care use and cost.[20,24] The data contained in this dataset are comparable to the data in validation datasets that included ICD-9-CM diagnosis and IP procedure codes, provider codes, Current Procedural Terminology (OP procedures) codes, National Drug Code, place and date of service, enrollment date(s), sex, year of birth, and US Census region. We report 95% confidence intervals (CIs)[25] for the ratio of the prevalence estimates of longer vs shorter intervals. Although these prevalence estimates are highly correlated, which would narrow these intervals substantially, we report uncorrected 95% CIs to be conservative.

Statistical analyses were conducted with either SAS version 9.4 (SAS Institute Inc, Cary, NC) or SPSS version 22 (IBM Inc, Armonk, NY).

Data availability

The datasets used in this study constitute secondary analyses of data from prior studies. Therefore, there are no data-sharing agreements in place for these data. The exception concerns the IMS data, which are a limited-use dataset provided gratis to University of Maryland, Baltimore faculty (W.J.C.), and data sharing is not allowed under that agreement.

Results

The characteristics of the 3 primary validation cohorts are representative of the general MS population (table 1).[5,6,10,13,26] As expected, there was a higher proportion of males in the VA data compared with KPSC and Manitoba, but the age distributions were similar across cohorts.

Initial validation of algorithm

Generally, when an algorithm was applied using 1 year of data, the sensitivity was lower than when applied using 2 years of data, while specificity was higher when applied using 1 year of data (table 3). The more stringent the case definition, that is, the greater the number of claims required, was, the lower the sensitivity was and the higher the specificity was. The addition of prescription claims for DMT to an algorithm increased sensitivity without a loss of specificity, leading to better performance overall.

Table 3

Summary of test statistics for each algorithm based on years of data for each data source

Algorithm MS_E ([IP + OP + DMT] ≥ 3) had the highest sensitivity for both time periods. In contrast, the algorithm with the highest specificity was MS_C (≥2 IP or ≥5 OP) in the 1-year and 2-year time periods. The PPVs were high for all algorithms, whereas the NPVs were moderately low, suggesting that the algorithms were better at ruling in cases of MS than ruling out non-MS cases (in a population with at least 1 claim for MS).

The proportion of misclassified cases (false-positives and false-negatives) was relatively low, as reflected in the moderately high to high accuracy scores; algorithm MS_E had the highest accuracy over both time periods. The Youden J statistic mirrored the crude accuracy scores, revealing a substantial level of agreement between each algorithm and the reference standard. Algorithm MS_E had the highest or second highest J statistic for the 1-year time period with scores of ≥0.60.

We performed the stratified analyses on only the algorithms that included DMTs because these were the best-performing definitions (MS_D and MS_E in tables 3). Because the stratified results were nearly identical across these 4 case definitions, we present only the results for MS_E-1 ([IP + OP + DMT] ≥ 3) for the 1-year time period (figures 1 and 2).

Figure 1

Performance of algorithm MS_E-1 [(IP + OP + DMT) ≥ 3] stratified by sex across the VA, KPSC, and MB datasets

(A) Men and women, (B) men only, and (C) women only.Data are presented as a proportion that can range between 0 and 1. DMT = disease-modifying therapy; IP = inpatient; KPSC = Kaiser Permanente Southern California; MB = Manitoba; MS = multiple sclerosis; NPV = negative predictive value; OP = outpatient; PPV = positive predictive value; VA = Department of Veterans Affairs.

Figure 2

Performance of algorithm MS_E-1 [(IP + OP + DMT) ≥ 3] stratified by age group across the VA, KPSC, and MB datasets

(A) Sensitivity, (B) specificity, (C) positive predictive value, (D) negative predictive value, (E) accuracy, and (F) Youden J statistic.Data from Manitoba (MB) for the 55- to 64- and 64- to 74-year age groups are suppressed due to small cell sizes. Data are presented as a proportion that can range between 0 and 1. DMT = disease-modifying therapy; IP = inpatient; KPSC = Kaiser Permanente Southern California; MS = multiple sclerosis; NPV = negative predictive value; OP = outpatient; PPV = positive predictive value; VA = Department of Veterans Affairs.

The test characteristics remained stable by sex (men, figure 1B; women, figure 1C), including sensitivity, specificity, PPV, and accuracy. In the VA, the NPV and J statistic were lower among women compared to men but were stable across sexes in the KPSC and Manitoba datasets. Similarly, sensitivity, PPV, and accuracy were relatively consistent across age groups and somewhat stable for specificity. The NPV and J statistic showed the greatest variation; however, despite this variability, the MS_E-1 algorithm performed well across datasets and age groups. Thus, given its consistent and superior performance, algorithm MS_E-1 was chosen as the preferred algorithm.

Validation in the general population

The characteristics of the Saskatchewan validation cohort were comparable to those of the other cohorts. When we applied the algorithms to the Saskatchewan general population, specificities and PPVs were all ≥0.96 and the NPVs were all ≥0.91. Of the algorithms using 1 year of data, algorithm MS_E-1 performed the best, along with algorithm MS_D-1. For preferred algorithm MS_E, the sensitivity was 0.96, specificity was 0.99, PPV was 0.99, NPV was 0.96, and Youden J was 0.96. One of the 2-year algorithms, MS_D-2, also had a specificity of 0.99 but a marginally higher sensitivity of 0.97.

Corrections for limited-duration ascertainment periods

Figure 3 compares estimated prevalence based on a 3-year (2008–2010) ascertainment period to that from 10 years (2000–2010) of data as of 2010 in the VA and Manitoba datasets. The underestimation of a 3-year ascertainment period ranged from 37% (95% CI 13%–66%) in the VA dataset to 47% (95% CI 23%–76%) in the Manitoba dataset. In the IMS (validation) dataset, the underestimation of a 3-year (2013–2015) ascertainment period compared to a 9-year period (2007–2015) as of 2015 was 39% (95% CI 13%–71%). The estimated prevalence of MS rose steeply up to about 2010 in all 3 datasets (figures 3 and 4) and then increased an average of 2.3%/y thereafter (2.1% in the VA, figure 3A; 2.5% in the IMS, figure 4).

Figure 3

Comparison of prevalence based on a 3- vs 10-year ascertainment period as of 2010 in the (A) VA and (B) MB datasets

CI = confidence interval; MB = Manitoba; VA = Department of Veterans Affairs.

Figure 4

Comparison of prevalence based on a 3- vs 9-year ascertainment period as of 2015 in the IMS (validation) dataset

CI = confidence interval; IMS = Intercontinental Marketing Services.

Discussion

We undertook this study to identify an algorithm that would accurately and consistently identify cases of MS across disparate AHC datasets representing different types of health care systems and geographic regions. The consistency of the findings across these datasets supports the generalizability of the algorithms to other AHC datasets in the United States when such validation studies are not feasible. The preferred algorithm (MS_E-1) required ≥3 encounters (IP + OP + DMT) for MS in a 1-year period. Algorithms that did not include DMT performed less well than those that did. However, we included the non-DMT algorithms to assess their ability to accurately identify cases of MS in studies in which pharmacy claims data might not be available; MS_A-1 performed nearly as well as the preferred algorithm (MS_E-1) and could be used when DMT is not available. Thus, by testing a variety of algorithms, our study provides value, and the findings can be applied in future research into the prevalence of MS across jurisdictions.

Algorithm MS E-1 ([IP + OP + DMT] ≥ 3) was preferred because of its consistent, high performance and ease of application. We hereafter refer to it as the MS algorithm. The 1-year algorithm readily allows annually updating of prevalence estimates in datasets with limited years of data. Sensitivity and PPV were high, consistent with previously published definitions developed with similar methods in the United States (VA) and Canada.[5,6,12,13] Specificity in our primary validation cohorts was modest, reflecting that our study cohorts were selected on the basis of having at least 1 diagnosis for MS. Tonelli et al.[27] evaluated algorithms to measure 40 comorbid conditions in AHC data from Calgary, Alberta, Canada. They defined an algorithm as valid if both sensitivity and PPV were ≥0.70 compared to a reference cohort. They did not consider specificity or NPV because they are generally ≥0.90 in the assessment of chronic disease in the general population. Consistent with these observations and recommendations, when the MS algorithm was applied to the general population in Saskatchewan, sensitivity (0.96), specificity (0.99), and PPV (0.99) markedly improved. These findings support our preferred algorithm as a valid method for identifying cases of MS in AHC datasets.

Minor variation in performance was observed when stratified by sex and age. However, accuracy remained high, and the Youden J statistic remained substantial. These findings indicate that the MS algorithm was an accurate and reliable algorithm for identifying cases of MS in these disparate datasets and outperforms similar algorithms for identifying amyotrophic lateral sclerosis[28] and Parkinson disease.[29]

When using AHC data, it is important for investigators to determine whether there is any hierarchy to the way that diagnosis codes are recorded because the performance of an algorithm can be adversely affected. In the VA, such a coding hierarchy exists, and when encounters with MS in any diagnosis position were used vs in the primary diagnosis position, specificity was compromised, as was the Youden J statistic. Surprisingly, accuracy was only slightly reduced but would likely be more adversely affected when applied to an entire dataset as opposed to the subset of cases with a diagnosis code for MS as used in this study.

Prior reports have defined cohorts in claims data based on a single encounter with MS listed as one of the diagnoses.[30,31] This strategy can result in the inclusion of a substantial number of false-positives. Prior work on the VA algorithm[5] showed that a rigorous algorithm (similar to the MS algorithm) ruled out 43% of the cases who had only a single MS encounter. This guided our decision to require multiple encounters as part of all of our algorithms. We strongly encourage other investigators to avoid the use of a single MS encounter to identify an MS cohort because, depending on the proportion of false-positives included, results could be substantially biased.

We found that prevalence estimates varied with the duration of the observation period in 3 different datasets (2 from the United States and 1 from Manitoba, Canada), suggesting that observation periods should be maximized to optimize case detection. Specifically, we found that an observation period of 3 years underestimated the prevalence of MS by 37% (95% CI 13%–66%) in the VA dataset and 47% (95% CI 23%–76%) in the Manitoba dataset. It should be noted that the 95% CI for these adjustment factors overlapped, implying that the growth range is nonsignificantly different for these 2 datasets. The findings in the IMS (validation) dataset corroborated results from the VA and suggest that a 3-year ascertainment period underestimates the 10-year prevalence in US datasets by ≈38% regardless of the final year of the time period of interest (2010 vs 2015). This is consistent with previous findings in Quebec, Canada, regarding systemic lupus erythematosus, a chronic disease like MS for which health care use may vary over time or with disease activity.[21] In that study, when a 3-year period was used, prevalence was underestimated by 66% compared with a 15-year period, while a 5-year period underestimated prevalence by 39%. Similar results have been reported for the assessment of the prevalence of cancer in Europe.[22] Registries with 5 years of data underestimated prevalence by 42% to 45%; as the duration of the registry increased, the underestimation of prevalence decreased systematically. For investigators without access to many years of data, these findings suggest that an adjustment factor should be applied to obtain a more stable and accurate prevalence estimate. The difference between the US (VA and IMS) and Manitoba datasets likely results because Manitoba has a universal health system with a single government payer and its AHC datasets capture >98% of the population,[6] whereas the US health care system involves multiple insurance payers, making it more difficult to achieve complete ascertainment of cases. Thus, the underestimation value of 37% provides a conservative (i.e., lower bound) inflation factor, whereas an inflation factor of 47% based on the Manitoba data likely better reflects what might be expected with nearly full capture of the US population over a 10-year period (i.e., upper bound).

This study included a limited number of AHC datasets that were restricted to North America, which may limit generalizability to datasets from other countries. However, the datasets that were included provided a diverse sampling of the MS population. The IMS validation dataset did not cover the same time period as the VA and Manitoba datasets; however, the 3-year underestimation in prevalence was nearly the same as the VA dataset despite differences in the time periods assessed. Our algorithms relied on ICD-9-CM coding (except for IP encounters in the Manitoba dataset, which used ICD-10-CM coding from 2004 onward) and have not been validated with ICD-10-CM coding. It is unlikely the algorithms will perform differently when ICD-10-CM coding is used because MS is still represented by a single 3-digit code.

We conducted a rigorous assessment of algorithms to identify cases of MS in several different AHC datasets. The algorithm of choice required the sum of ≥3 MS-related IP admissions, MS-related OP encounters, or pharmacy claims for a DMT within a calendar year. Furthermore, this algorithm performed remarkably well across different datasets and demographic groups. We propose that this MS algorithm be adopted for future MS-related studies using AHC datasets to enhance comparability across studies.