Validation of the Swedish Multiple Sclerosis Register: Further Improving a Resource for Pharmacoepidemiologic Evaluations.

The Swedish Multiple Sclerosis Register is a national register monitoring treatment and clinical course for all Swedish multiple sclerosis (MS) patients, with high coverage and close integration with the clinic. Despite its great value for epidemiologic research, it has not previously been validated. In this brief report, we summarize a large validation of >3,000 patients in the register using clinical chart review in the context of the COMBAT-MS study. While further improving the data quality for a central cohort of patients available for future epidemiologic research, this study also allowed us to estimate the accuracy and completeness of the register data.

Registers following patients in clinical practice, such as the Swedish Multiple Sclerosis (MS) Register,[1] are valuable and frequently used data sources in studies of long-term effectiveness, safety, and tolerability of therapies in unselected patient populations.

The Swedish MS Register has provided data for over 100 scientific reports[2] (recent examples[3-12]), yet its data have never been formally validated. Coupled with the reliance on voluntary data entry collected as part of clinical practice, this raises concerns about the accuracy and completeness of data, which, if varying by treatment, may potentially bias comparative effectiveness and safety studies.

As part of the COMparison Between All immunoTherapies for Multiple Sclerosis study (COMBAT-MS; clinicaltrials.gov, NCT03193866), we performed a comprehensive clinical chart review of a central cohort of >3,000 patients to validate and update missing or erroneous information in the register. The COMBAT-MS study is approved by the regional ethical review board in Stockholm (2017/32-31/4).

METHODS

The Swedish MS Register is a publicly funded national healthcare register. Since its launch in 2000, it has become well integrated in the clinical documentation at Sweden’s neurology clinics.[1,2] Participation in the register is voluntary for both patients and neurologists, with no reimbursements linked to data entry. Nevertheless, coverage has reached almost 80% of the prevalent Swedish MS population,[2] with ~17,000 active patients.[1] Data are recorded by physicians or nurses through an electronic interface and include patient characteristics, MS disease data, therapies, visits, clinical scales (e.g., Expanded Disability Status Scale [EDSS]), relapses, magnetic resonance imaging (MRI), and laboratory tests.[1] Most data are collected at routine clinical visits, but relapses and MRI are recorded at the time of event. Additional data can be retrieved by linking the MS register to the nationwide system of Swedish compulsory healthcare and demographic registers (see examples[9-12]).

Study Population

Patients were identified through the MS register using the following criteria:

Treated at any Swedish university clinic;

Starting a first or second therapy after 1 January 2011 (the inclusion therapy); and

Relapsing-remitting MS at the start of the inclusion therapy.

Therapies considered were rituximab, fingolimod, natalizumab, dimethyl fumarate, alemtuzumab, teriflunomide, mitoxantrone, interferon beta-1a, interferon beta-1b, peginterferon beta-1a, glatiramer acetate, and hematopoietic stem cell transplantation.

A switch between injectables (interferons and glatiramer acetate) was considered a single therapy with regards to the inclusion therapy.

Clinical Chart Review

Lists of patients and standardized instructions for the clinical chart review (eAppendix 1; http://links.lww.com/EDE/B436) were distributed to the clinics. Clinics were instructed to add or correct any missing or erroneous data in the register for patient and disease information, therapies, EDSS and other scores, clinical relapses, and MRI (original radiology report). The focus on EDSS and MRI data was motivated by the utility of these disease activity measures, both for clinical decision making and as primary outcomes in drug trials. If there was a conflict between chart data and the register, clinics were instructed to update the register using the chart data as the reference. Sites were reimbursed per patient chart reviewed to motivate high compliance.

Statistical Methods

We extracted data from the register before and after the COMBAT-MS update (9 January 2017 and 21 November 2017, respectively). Data were restricted to patients existing in both datasets and to observations before 1 January 2017, to only capture changes made through the update. For each type of observation, an identifier and data variables were specified.

We compared data on therapies, rituximab infusions, relapses, MRI, and EDSS pre- and postupdate to identify observations that were changed (same identifier, changed data), removed (identifier not present postupdate), or added (identifier not present preupdate).

Descriptive statistics for the pre- and postupdate number of therapy episodes, as well as number of relapses, values of EDSS, and proportion of MRIs reporting contrast-enhancing lesions, within 3 years of therapy start, were tabulated stratified by therapy. We also compared the proportions with at least one valid EDSS and MRI, respectively, at therapy start (EDSS −180 to +30 days; MRI −90 to +30 days).

To identify the strongest predictors of having preupdate missing data on EDSS and MRI at treatment start, we used logistic regression models with Akaike information criterion (AIC)–based backward selection among available covariates. To reduce variability, these analyses were run for rituximab, fingolimod, and natalizumab only (the dominant second-line therapy options).

RESULTS

In total, 3,012 patients were identified as updated in COMBAT-MS and included in the analyses. Differences in observations, between pre- and postupdate, of therapy, rituximab infusions, relapses, MRI, and EDSS are summarized in Table 1 (expanded contingency and accuracy measures in eTables 1 and 2, respectively; https://links.lww.com/EDE/B436.). Few observations had been changed (≤7%) or removed (≤3%) for all categories except MRI (34% changed). Added observations ranged from an increase of 5% (therapy) to 71% (MRI). Different clinical centers (regions) had similar (high) accuracy in recorded variables but varied greatly in missing (i.e., nonrecorded) data, in particular for rituximab infusions, relapses, and MRI results (eTables 3 and 4; http://links.lww.com/EDE/B436).

TABLE 1.

Number of Observations Before and Changed/Removed/Added After Chart Review

Although overall observations of relapses increased by 35%, most additions were before treatment start; increase in relapses within 3 years after treatment start was modest, corresponding to a sensitivity just below 80% and a specificity above 99% (Table 2 and eTable 2; http://links.lww.com/EDE/B436). The relative increase in MRIs with contrast-enhancing lesions followed the same pattern, with similar specificity although with lower sensitivity (just above 50%). When recorded, EDSS values were very accurate, 0.9% (n = 166) differed between register and chart and only 0.06% (N = 10) did so by more than 2.5 EDSS units (eTable 5; http://links.lww.com/EDE/B436).

TABLE 2.

Differences in Value Between Preupdate Register and Chart Data

The proportion of therapy episodes with an associated EDSS and MRI observation at therapy start, pre- and postupdate, are depicted in the Figure. Increases in proportions were seen in the postupdate data for both EDSS and MRI in all therapy categories. However, after chart abstraction, baseline EDSS and MRI were still missing in 18%–25% and 27%–42% of therapy starts, respectively, for rituximab, fingolimod, and natalizumab. For injectable therapies this figure was higher, 45% missing EDSS and 58% missing MRI.

Proportion of therapy starts with an observation of EDSS (left) and MRI (right), within −180 to +30 days (EDSS) and −90 to +30 days (MRI) of therapy start, stratified by therapy. EDSS, Expanded Disability Status Scale; FGL, fingolimod; INJ, injectables; MRI, magnetic resonance imaging; NTZ, natalizumab; other, all other therapies; RTX, rituximab.

Regression models indicated that the most important factors associated with missing data in the register were (in descending order of AIC), for EDSS: region, number of therapies, therapy, and sex; and for MRI: region, therapy, and age (eTables 6 and 7; http://links.lww.com/EDE/B436).

DISCUSSION

This report summarizes the results of a large systematic update and validation of the Swedish Multiple Sclerosis Register, a frequently used resource for epidemiologic research in MS.

There is increasing awareness that long-term postmarketing studies of real-world patient populations are needed to supplement the limited safety and effectiveness data available from the pivotal trials. Even more striking is the need for monitoring of off-label use of drugs, such as rituximab, where data from randomized trials are sparse or not available at all. Large clinical registers such as the Swedish MS register thus have an important role in this context. However, the validity of results derived from such registers is often limited by missing data, unknown data quality, possible selection bias in inclusion or missingness pattern, and the availability of covariates to control for confounding by indication. As the Swedish MS register is a nearly population-based register, selection bias is not a major issue; however, the completeness and quality of data had not previously been addressed.

Comparing data entered into the register voluntarily by clinicians, with patients’ medical records as reference, the register data on treatment exposure and EDSS were of acceptable completeness. In contrast, MRI data were often missing or incomplete. We also found that clinicians were less likely to have documented an EDSS or obtained an MRI at therapy start with older injectable therapies compared with newer therapies. These discrepancies, together with the differences between regions, underscore the importance of data validation in registers that require data entry separated from the clinical records systems.

The substantial increase in observations of MRIs and clinical relapse episodes improve the data quality for future comparative effectiveness research, mainly by improving the ability to account for confounding by indication. Most previously unrecorded relapses occurred before the start of therapy, leaving the number of relapses after therapy start relatively stable. Similarly, despite the high proportion of previously unrecorded MRIs, the increase in the number of contrast-enhancing lesions observed after treatment start was modest. The increased data quality is thus of greatest value for baseline covariates, rather than outcome measures, and missingness should not have substantially biased previous studies of these endpoints.

Rituximab infusions and EDSS also received additional observations. The added infusions reflect the ongoing effort to register all rituximab infusions given at the participating clinics. For EDSS, mean values did not change after the addition of the missing observations, suggesting they were mostly missing at random, and imputation methods (e.g., multiple imputation) may be suitable to deal with the missing EDSS data in the nonupdated cohort.

Therapy starts with valid EDSS and MRI increased after the update but did not reach 100% and remained notably low for injectables (55% and 42%, respectively), indicating differing follow-up routines in clinical practice across therapies and treatment centers.

In summary, this update improved the data quality for a central cohort of patients in the register and provided an indication of the accuracy and completeness in the remaining cohort, although care is needed when generalizing due to the differences between regions. This impacts future research by providing a measure of validity for a core part of the register, reducing the need for complementary clinical chart review and further increasing the value of the Swedish MS register as a resource for pharmacoepidemiologic studies in MS.

ACKNOWLEDGMENTS

The members of the COMBAT-MS study group are as follows: Peter Alping, MSc, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden; Joachim Burman, MD, PhD, Department of Neuroscience, Uppsala University, Uppsala, Sweden; Katharina Fink, MD, PhD, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden; Anna Fogdell-Hahn, PhD, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden; Thomas Frisell, PhD, Clinical Epidemiology Division, Department of Medicine Solna, Karolinska Institutet, Stockholm, Sweden; Martin Gunnarsson, MD, PhD, Center for Health and Medical Psychology, Örebro University, Örebro, Sweden; Jan Hillert, MD, PhD, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden; Ingrid Kockum, PhD, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden; Annette Langer-Gould, MD, PhD, Clinical and Translational Neuroscience, Southern California Permanente Medical Group, Kaiser Permanente; Jan Lycke, MD, PhD, Department of Clinical Neuroscience and Rehabilitation, University of Gothenburg, Gothenburg, Sweden; Petra Nilsson, MD, PhD, Department of Clinical Sciences/Neurology, Lund University, Lund, Sweden; Tomas Olsson, MD, PhD, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden; Fredrik Piehl, MD, PhD, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden; Jonatan Salzer, MD, PhD, Department of Pharmacology and Clinical Neuroscience, Umeå University, Umeå, Sweden; Anders Svenningsson, MD, PhD, Department of Clinical Sciences, Danderyd Hospital, Karolinska Institutet, Stockholm, Sweden; Suvi Virtanen, MSc, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden; and Magnus Vrethem, MD, PhD, Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden.