Using Medicare Claims to Identify Acute Clinical Events Following Implantation of Leadless Pacemakers.

BACKGROUND: There is heightened interest in how real-world data (RWD) can be used to supplement or replace traditional mechanisms for collecting clinical information. A critical component in evaluating utility of RWD is assessing the validity and reliability of event measurement. Only two studies have validated Medicare claims with physician-adjudicated data collected in a clinical study and none in the pacemaker patient population. This study compares events identified in physician-adjudicated clinical registry data collected in the Micra Post-Approval Registry (PAR) with events identified via Medicare administrative claims in the Micra Coverage with Evidence (CED) Study.
METHODS: Patients who were dually enrolled in the Micra CED and the Micra PAR between March 9, 2017 and December 1, 2017 were included in the validation analysis. All patients intended to be implanted with a Micra device were eligible for participation in the Micra PAR. All Medicare fee-for-service beneficiaries implanted with a Micra device who met the 12-month continuous enrollment criteria were included in the Micra CED. We compared the count of acute (30-day) complications identified in the Medicare claims and the physician-adjudicated PAR data to assess agreement between data sources.
RESULTS: There were 230 patients dually enrolled in the Micra CED and Micra PAR studies during the study period. Overall, there were 17 acute events reported in either the Micra CED or the Micra PAR, with 95% agreement in the identification of events and absence of events between studies. Study disagreement between events reported in either study varied: arteriovenous fistula (50%), pulmonary embolism (67%), hemorrhage/hematoma (75%), and deep vein thrombosis (100%). Among physician-adjudicated events, there was no disagreement between the Micra CED and Micra PAR studies in any event type.
CONCLUSION: Findings from this study demonstrate high agreement in event identification between Medicare claims data and registries for patients implanted with Micra leadless pacemakers.

Introduction

Prospective interventional studies using randomization, blinding, and controls are the gold standard for measuring the safety and efficacy of medical interventions and therapies.1 These studies rely on primary data collection and often use physician-adjudication committees to independently attribute clinical events to the intervention under study and determine event severity. However, these studies may not answer questions related to an intervention’s safety or effectiveness in a generalizable population due to narrow patient selection criteria, research site specialization, limited follow-up periods, and ethical considerations. Healthcare stakeholders (e.g. regulators, payers, industry) are increasingly interested in how to use real-world data (RWD) to develop generalizable evidence on the safety and effectiveness of interventions. For example, the United States Food and Drug Administration (FDA) issued specific guidance in 2017 on using RWD to support post-market safety studies, label expansion, and clinical trial design in the medical device industry.2 Accordingly, recent efforts by the medical device industry have combined administrative claims and/or registry data with clinical trial data to demonstrate safety and efficacy of device interventions.3

Real-world data are defined as data related to patient health status or the routinely collected healthcare data from a variety of sources, including electronic health records, claims and billing data, product and disease registries, and patients. While RWD includes a broad spectrum of sources, there has been considerable attention on how non-interventional studies using secondary RWD, or data collected for a purpose other than research, can develop reliable evidence to support healthcare decision-making. Non-interventional studies using secondary RWD, such as claims and billing data, typically include broader patient selection criteria, and offer longer follow-up of patient care and outcomes than interventional studies using primary data collection. However, these benefits are frequently tempered by biases introduced via treatment selection, missing data, measurement error, and confounding of observed and unobserved characteristics.

Regardless of the statistical methods applied to adjust for biases associated with RWD, studies using secondary data are typically unable to rely on a physician-adjudication committee to attribute the events observed because of the nature of data collection. If the underlying RWD used to measure and attribute events are not valid or reliable, the resultant study inferences and conclusions will be flawed. Therefore, efforts to assess the validity of RWD are imperative.

A recent example of a non-interventional study using secondary RWD is the Micra Coverage with Evidence Development (CED) Study. The Micra CED Study was designed for the purposes of Medicare coverage and evaluates the Micra Transcatheter Pacemaker System, a miniaturized leadless ventricular pacemaker system implanted directly into the right ventricle of the heart. The Micra CED Study is a prospective, longitudinal study that relies on administrative claims data of the Medicare fee-for-service (FFS) population to enroll patients, identify patient characteristics, comorbidities, and events. In addition to the Micra CED Study, the Micra Post-Approval Registry (PAR) (a global, multi-center registry using primary data collection and a physician-adjudication committee to classify the relatedness and severity of any reported adverse events to the Micra system) is being conducted to further confirm the safety and effectiveness of the Micra leadless pacemaker system when used as intended, in real-world clinical practice.4 The Micra CED and the Micra PAR are being conducted in parallel, and thus allow for the unique research opportunity to assess the validity of claims and billing data to identify pacemaker-related events.

To our knowledge, only two studies have validated Medicare claims with physician-adjudicated data collected in a clinical study and none have been conducted in the pacemaker patient population.5,6 To assess the reliability and validity of claims and billing data to identify pacemaker-related events, we linked the Micra CED Study to the Micra Post-Approval Registry and compared the events identified from Medicare claims data to the physician-adjudicated events from the PAR.

Methods

Study Population

Patients indicated for Micra leadless pacemakers have (1) symptomatic paroxysmal or permanent high-grade atrioventricular block (AV) in the presence of atrial fibrillation (AF), (2) symptomatic paroxysmal or permanent high-grade AV block in the absence of AF, as an alternative to dual-chamber pacing, when atrial lead placement is considered difficult, high risk, or not deemed necessary for effective therapy, or (3) symptomatic bradycardia-tachycardia syndrome or sinus node dysfunction (sinus bradycardia or sinus pauses), as an alternative to atrial or dual-chamber pacing, when atrial lead placement is considered difficult, high risk, or not deemed necessary for effective therapy.

The Micra CED study is a continuously enrolling prospective cohort study that includes all Medicare FFS beneficiaries implanted with a Micra leadless pacemaker beginning on the date of Centers for Medicare and Medicare Services (CMS) coverage approval (March 9, 2017). In order to assess patient characteristics and comorbidities, patients are required to have a minimum of 12 months of continuous enrollment in Medicare Part A and B prior to the date of Micra implant. There are no additional exclusion criteria applied in the Micra CED Study. Procedure codes for leadless pacemakers (e.g. International Classification of Disease, Tenth Edition (ICD-10), Current Procedural Terminology (CPT)) are unable to identify specific device models (e.g. Micra). Therefore, the Micra CED study identifies Micra leadless pacemaker patients with a two-step approach combining patient information included in manufacturer device registration data with Medicare claims and enrollment data. First, a CMS contractor links device registration and Medicare data with an exact match on patient identifiers including social security number, first name, last name, date of birth, patient sex, and zip code. The exact match process does not allow flexibility in minor discrepancies between data sources (e.g. “Tom” versus “Thomas”). During the study period, the Micra leadless pacemaker was the only leadless pacemaker FDA approved, covered by CMS, and using the leadless pacemaker procedure code. The contractor linking approach matched 73% of Medicare FFS patients with a claim for leadless pacemaker implant to the device registration data.

A second linking approach is applied to all claims with a procedure code for leadless pacemaker implant (CPT 0387T or 33274 or ICD-10 PCS 02HK3NZ) that are not linked by the CMS contractor approach. The secondary linking approach conducted by the manufacturer includes all remaining unlinked claims for leadless pacing with a National Clinical Trial number indicating the Micra CED Study; all remaining unlinked claims where implant date on the registration and claim are allowed to differ by 7 days, date of birth is allowed to differ by 2 days, and a perfect match on patient sex; and all remaining unlinked claims that are linked to manufacturer complaint handling data. The combined data linking approach matched 84% of patients with a Medicare claim for leadless pacemaker implant to the manufacturer registration data. An analysis of the combined linking approach identified a 99.6% sensitivity of the secondary linking approach among the subset of patients identified in the CMS contractor linking process. The Micra CED study protocol was approved by the Western International Review Board and is conducted under a waiver of HIPAA.

The Micra PAR is an FDA-approved global, prospective, observational, multi-center study to further confirm the safety and effectiveness of the Micra leadless pacemaker system when used as intended, in real-world clinical practice.4 All patients intended to be implanted with a Micra device were eligible for participation in the PAR. Following consent, baseline, medical history, and implant information were collected. Patients in the PAR are followed per their institution’s standard of care and all adverse events potentially related to the Micra system or procedure are collected and assigned a standardized diagnosis using the Medical Dictionary for Regulatory Activities (MedDRA version 22.0). A more detailed review of the PAR enrollment procedure is described elsewhere.7

Patients were included in the validation analysis if they were enrolled in both the Micra CED Study and Micra PAR and implanted with Micra between March 9, 2017 and December 1, 2017. Micra CED and Micra PAR data were linked via device serial number. The Micra PAR records the device serial number for all enrolled patients. The Medicare claims data do not include device serial number; device serial number is appended to the claims during the two-step linking process. Patients enrolled in a commercial Medicare Advantage plan were not included because Medicare claims data were only available for FFS enrollees.

Data Sources

The PAR has enrolled 1817 patients between July 2015 and March 2018 and the required 9-year follow-up period is currently ongoing. In this analysis, data from the Micra PAR included baseline and medical history from all patients with a successful Micra implant in the United States between March 9, 2017 and December 31, 2017. In addition, all adverse events adjudicated as related to the Micra system or procedure by the independent physician committee were obtained for the subset of Micra CED-PAR linked patients.

For all Micra CED Study patients and Micra CED-PAR linked patients, we used Medicare inpatient, outpatient, and carrier claims and enrollment files from calendar years 2016 to 2018. The claims files included procedure and encounter/hospitalization dates, ICD-10-CM diagnosis codes, ICD-10-CM, and CPT/Healthcare Common Procedure Coding System procedure codes. The enrollment files included patient information such as dates of birth and death, sex, and Medicare program enrollment.

Outcomes

We can validate complication measures where there is consistency between what is being measured in the Micra CED and the Micra PAR studies and the consistent measures are observable in the administrative claims. Based on a review of the acute (30-day) complications reported individually from both studies, it is reasonable to assess the measurement validity of the following system or procedure-related complications: embolism and thrombosis, cardiac effusion or perforation, pericardial effusion, device dislodgement, infection, and serious groin complications (e.g. arteriovenous fistula, hematoma, hemorrhage). Although 30-day all-cause mortality is not an objective in either study, all-cause mortality at any follow-up time point is captured in both studies; therefore, we included 30-day all-cause mortality as a measure in this analysis.

Table 1 defines the ICD-10-CM diagnosis codes used to identify complications in the Medicare claims. For all events measured in the Micra CED Study, ICD-10 codes may appear in any position on a claim and on all claim types (e.g. inpatient, outpatient, and carrier) from the index implant date through 30 days after the index implant date. To attribute a complication to a patient during the index implant encounter, the present on admission (POA) indicator associated with an ICD-10 diagnosis code for a pre-specified study complication must indicate the diagnosis was not POA. All complications occurring after the index implant encounter are counted as events regardless of the POA indicator value.

Table 1

Event ICD-10-CM Diagnosis Codes

Event Measure	ICD-10-CM Definitions
Deep vein thrombosis	I82.401 or I82.402 or I82.403 or I82.409 or I82.411 or I82.412 or I82.413 or I82.419 or I82.421 or I82.422 or I82.423 or I82.429 or I82.4Y1 or I82.4Y2 or I82.4Y3 or I82.4Y9 or I82.4Z1 or I82.4Z2 or I82.4Z3 or I82.4Z9 or I82.621 or I82.622 or I82.623 or I82.629 or I82.A11 or I82.A12 or I82.A13 or I82.A19
Pulmonary embolism	I26.01 or I26.02 or I26.09 or I26.90 or I26.92 or I26.99
Arteriovenous fistula	I77.0
Cardiac perforation	I97.51
Pericardial effusion	I97.51 + (I30.9 or I31.3)
Cardiac tamponade	I31.4
Device dislodgement or displacement of cardiac electronic device	T82.12++
Infection and inflammatory reaction due to other cardiac and vascula devices, implants and grafts	T82.7+++
Hematoma - post procedural	I97.638
Hemorrhage - post procedural	I97.618

Abbreviation: ICD-10-CM, International Classification of Disease, Tenth Edition.

Events in the Micra PAR including groin access issues (e.g. arteriovenous fistula, incision site hemorrhage, incision site hematoma), pericardial effusion (including cardiac perforation and tamponade), device dislodgement, and infections (e.g. localized or systemic) are identified based on verbatim event diagnosis and MedDRA preferred term coding based on the verbatim event diagnosis recorded on case report forms.

Adverse events in the PAR are adjudicated for their relationship to the Micra system or procedure by an independent committee of physicians at regular intervals. Adverse events classified as related are to the system or procedure are further adjudicated for severity (major complication, minor complication, or observation). Major events include events that result in hospitalization, hospitalizations that have been prolonged by 48 hrs or more, permanent loss of device function, or death; minor events include events that require invasive therapy (e.g. intravenous drugs) but are not major; and observations are events that do not require an invasive intervention. The primary acute performance outcomes from the Micra PAR are reported elsewhere.7,8

Statistical Analysis

Baseline and medical histories were compared using t-tests (continuous variables) or the Fisher Exact test (categorical variables) between PAR patients enrolled in the United States between March 9, 2017 and December 31, 2017 who were co-enrolled in the CED with those who were not co-enrolled in the CED.

We compared the count of acute complications identified in the Medicare claims and the physician-adjudicated PAR data to assess agreement between data sources. In order to protect beneficiary privacy, CMS prohibits reporting on any cell value with an aggregate value between one and ten.13 Additionally, the reporting of any cell value that allows back calculation of another cell with an aggregate value between one and ten is prohibited. Due to the small event rates observed in the Micra CED and Micra PAR studies and data suppression requirements imposed by CMS, we present results as the rounded percent of event agreement and disagreement identified in both studies. We present three measures of agreement between the Micra CED and Micra PAR studies. First, we present overall event agreement between the Micra CED and Micra PAR among all dually enrolled patients, as typically represented by the on-diagonal cells in a contingency table. Second, we present event disagreement among all events identified in both studies. Third, we present event disagreement among the subset of physician-adjudicated events identified in the PAR. All analyses were conducted in SAS V9.4 and Microsoft Excel.

Results

Patient Characteristics

A total of 1757 Medicare FFS patients who met the CED study enrollment criteria were implanted with a leadless pacemaker between March 9, 2017 and December 1, 2017. During the same time period, 491 US patients were enrolled in the PAR. Among all US patients enrolled in the Micra CED and Micra PAR during the study period, 230 were dually enrolled in both studies and thus included in this study (Figure 1). Table 2 shows the patient demographic and clinical characteristics of the Micra PAR patients who were and were not dually enrolled in the Micra CED. Dually enrolled patients tended to be older (78.8±9.9 versus 74.2±13.6, P<0.001) and have higher rate of atrial fibrillation and hypertension compared to those PAR patients not enrolled in the CED.

Figure 1

Study flow chart.

Abbreviations: CED, Coverage with Evidence Development; FFS, fee-for-service; PAR, post-approval registry.

Table 2

Baseline Characteristics

Patient Characteristic	Patients Enrolled in Micra CED and Micra PAR Studies (N = 230)	US PAR Patients Not Enrolled in the CED (N=261)	P-value
Demographics
Age at implant, mean ± SD	78.8 ± 9.9	74.2 ± 13.6	<0.001
Female, n (%)	95 (41.3%)	114 (43.7%)	0.58
Comorbidities
CHF, n (%)	60 (26.1%)	67 (25.7%)	1.00
COPD, n (%)	32 (13.9%)	23 (8.8%)	0.09
Diabetes, n (%)	75 (32.6%)	84 (32.2%)	1.00
Renal dysfunction, n (%)	59 (25.7%)	60 (23.0%)	0.53
Atrial fibrillation, n (%)	195 (84.8%)	193 (73.9%)	0.002
Hypertension, n (%)	192 (83.5%)	191 (73.2%)	0.008
Treatments
Prior CIED	25 (10.9%)	34 (13.0%)	0.49
Dialysis, n (%)	25 (10.9%)	33 (12.6%)	0.58

Abbreviations: CED, coverage with evidence development; PAR, post-approval registry; CIED, cardiac implantable electronic device; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease.

Agreement Between Studies

Table 3 displays the agreement and disagreement statistics comparing events in the Micra CED and the Micra PAR studies. Overall, there were 17 acute events reported in either the Micra CED or the Micra PAR, with 95% agreement in the identification of events and absence of events between Medicare claims and physician-adjudicated data. There was perfect agreement between the Micra CED and Micra PAR studies in identification of presence or absence of effusion/perforation, dislodgement, infection (not involving the device), and 30-day all-cause mortality. Notably, there were no dislodgement events identified in either study; therefore, the 100% agreement statistic represents perfect identification of absence of events only. Less than perfect agreement ranged from 98% for the presence or absence of deep vein thrombosis and hemorrhage/hematoma to 99% for the presence or absence of pulmonary embolism, and arteriovenous fistula.

Table 3

Agreement and Disagreement Between Events Reported in the Micra CED and Micra PAR Studies

Acute (30-Day) Event	Agreement Between Events and No Events (%)	Disagreement Between Events (%)	Disagreement Between Physician-Adjudicated Events (%)
All Events	95%	65%	0%
Effusion/Perforation	100%	0%	0%
Deep Vein Thrombosis	98%	100%	N/A
Pulmonary Embolism	99%	67%	0%
Arteriovenous Fistula	99%	50%	0%
Dislodgement	100%	N/A	N/A
Infection	100%	0%	0%
Hemorrhage/Hematoma	98%	75%	0%
All-cause 30-day mortality	100%	0%	0%

Abbreviations: CED, coverage with evidence development; PAR, post-approval registry.

Disagreement Between Studies

Disagreement conditioning on the events reported in at least one of the studies is shown in Table 3. Overall, we observed 65% disagreement between events identified in either the Micra CED or the Micra PAR. Because there were no events identified in either study, we do not report a disagreement statistic for dislodgments. Study disagreement between events reported in either study varied by event type: arteriovenous fistula (50%), pulmonary embolism (67%), hemorrhage/hematoma (75%), and deep vein thrombosis (100%). Among physician-adjudicated events, there was no disagreement between the Micra CED and Micra PAR studies in any event type.

Discussion

There is heightened interest in how RWD can be used to supplement or replace traditional mechanisms for collecting clinical information. A critical component in evaluating utility of RWD is assessing the validity and reliability of event measurement. Several studies have assessed the validity of administrative claims compared to medical charts and electronic health records, but results are mixed depending on the outcome or condition under evaluation, source of the claims data, and the method of analysis.9–13 Lowenstern et al found strong agreement between Medicare claims and adjudicated clinical registry data in a comparison of adverse events after transcatheter mitral valve repair.5 Hlatky et al found that Medicare claims-based definitions of acute myocardial infarction and coronary revascularization had good or excellent agreement with data collected in two RCTs included in the Women’s Health Initiative.6 While these studies indicate strong validity in the Medicare claims compared to physician-adjudicated data, the generalizability of these findings to other conditions, procedures, and events are unknown. In this study, where we evaluated the event measurement validity for patients implanted with Micra leadless pacemakers enrolled in both a Medicare claims data study (Micra CED Study) and the physician-adjudicated post-approval registry (Micra PAR), we have several findings.

First, our findings are generally consistent with two previous studies that described good to strong agreement between administrative claims and physician-adjudicated clinical study data for the identification of events following transcatheter mitral valve repair, acute myocardial infarction, and coronary revascularization procedures.5,6 Because our study focused on clinical events relevant to leadless pacemakers, the majority of the events evaluated have not been assessed in prior studies. Overall, the Micra CED and PAR studies were 95% in agreement in their identification of acute complications and 100% in agreement for 30-day all-cause mortality. All the major and minor adverse events observed in the PAR data were accurately identified in the CED study data.

Second, claims data may overestimate certain events due to lack of procedure or diagnosis code specificity. Similar to findings from Kucharska-Newton et al who identified strong agreement between claims and medical discharge records but potential overestimation of heart failure hospitalizations in the Medicare claims,13 our analysis indicates varying strength of agreement across the complications included in this study. We found perfect agreement in identification of acute cardiac effusion and perforation, device dislodgement, infection (not involving the device), and 30-day all-cause mortality between the Micra CED and Micra PAR studies. The diagnosis codes used to identify cardiac effusion and perforation and device dislodgement include cardiac- or device-specific language that facilitate direct attribution of an event to a device or device-related procedure. We expected excellent agreement for 30-day all-cause mortality as its validity has been previously demonstrated in Medicare claims data.5 Disagreement among the remaining events (deep vein thrombosis, pulmonary embolism, arteriovenous fistula, and hemorrhage/hematoma) ranged from 50% to 100%. All cases of event disagreement were due to an event identified via ICD-10 diagnosis code in the Medicare claims, but adjudicated as not related to the Micra system or procedure in the clinical study data. For example, the diagnosis codes used to identify deep vein thrombosis, and hemorrhage or hematoma events do not specify anatomical site or device, thus making direct attribution to the Micra device or implant procedure difficult as patients may have multiple un-related interventions resulting in a hematoma from between Micra implant and date the event occurred.

Although we anticipate a difference in the event rates reported in the Micra CED and the clinical studies of leadless pacing, comparative effectiveness measures should not be influenced by mis- or over-attribution in the administrative claims. It is reasonable to assume there is no difference in the procedure and diagnosis coding process between patients who receive leadless or traditional pacing therapy. If overestimation is not different between study arms, the relative difference between the arms should not be impacted. Therefore, measures of effectiveness comparing the two technologies should not be adversely affected by overestimation of events that may occur in the Medicare claims provided proper techniques for risk-adjustment are utilized to balance potential differences in baseline co-morbidity.

This study adds to the important and growing body of literature evaluating the validity of administrative claims data compared to physician-adjudicated data collected in traditional clinical studies. It is also the first study to measure agreement in the Medicare fee-for-service pacemaker population. Using RWD such as Medicare claims as the underlying source to generate evidence has the advantage of providing an uncontrolled setting in which all patients who receive a treatment, not only those eligible for a clinical study, are observed. Yet using secondary RWD as the source to develop evidence on an intervention’s safety and effectiveness requires both a thoughtful approach to analysis and assurance that the underlying data are a valid source for measuring events of interest. As the use of RWD for meeting regulatory requirements in the medical device industry grows, efforts to validate event identification is a critical component of assessing whether the data is fit for purpose.

There are some limitations to this study. Our study is limited to 30-day acute complications. If there is a difference in the validity of claims data by time from the index event our results may misrepresent the agreement between physician-adjudicated registry data and Medicare claims data over a longer time period. The matching process used to identify patients with a Micra leadless pacemaker relies on patients being registered in the manufacturer’s registration database. We are unable to identify unsuccessful implant attempts because the devices would not be registered. This analysis evaluated agreement among rare events, precluding a statistical analysis of agreement between registry and Medicare claims data. However, our summary measures of agreement and disagreement provide satisfactory evidence of event identification between Medicare claims compared to clinically adjudicated data. Our study population is limited to patients dually enrolled in the Micra PAR and Micra CED studies. The dually enrolled patients are significantly different in at least some clinical characteristics than those only enrolled in the PAR, and thus our results may not be generalizable to the entire Micra patient population. Additionally, institutions that participate in the Micra PAR may have different coding and billing practices than institutions that do not participate in the Micra PAR, thus our results may not be generalizable to all institutions implanting leadless pacemakers.

Conclusion

Manufacturers, regulatory agencies, and patients all stand to benefit from evidence generation that brings safe and effective healthcare treatments to the market quickly and efficiently. In the medical device industry, RWD are increasingly being considered as an alternative to primary data collection in clinical studies and as a source to develop evidence on an intervention’s safety and effectiveness. Findings from this study demonstrate high agreement in event identification between Medicare claims data and registries for patients implanted with Micra leadless pacemakers. However, further research is necessary to understand the areas Medicare claims data may overestimate events to inform the utility of administrative claims data for use in generating reliable evidence to be used in regulatory and payer decision-making.