Validating Acute Myocardial Infarction Diagnoses in National Health Registers for Use as Endpoint in Research: The Tromsø Study.

PURPOSE: To assess whether acute myocardial infarction (MI) diagnoses in national health registers are sufficiently correct and complete to replace manual collection of endpoint data for a population-based, epidemiological study. PATIENTS AND METHODS: Using the Tromsø Study Cardiovascular Disease Register for 2013-2014 as gold standard, we calculated correctness (defined as positive predictive value (PPV)) and completeness (defined as sensitivity) of MI cases in the Norwegian Myocardial Infarction Register and the Norwegian Patient Register separately and in combination. We calculated the sensitivity and PPV with 95% confidence intervals using the Clopper-Pearson Exact test.
RESULTS: We identified 153 MI cases in the gold standard. In the Norwegian Myocardial Infarction Register, we found a PPV of 97.1% (95% confidence interval (CI) 92.8-99.2) and a sensitivity of 88.2% (95% CI 82.0-92.9). In the Norwegian Patient Register, the PPV was 96.3% (95% CI 91.6-98.8) and the sensitivity was 85.6% (95% CI 79.0-90.8). The combined dataset of the Norwegian Myocardial Infarction Register and the Norwegian Patient Register had a PPV of 96.6% (95% CI 92.1-98.9) and a sensitivity of 91.5% (95% CI 85.9-95.4).
CONCLUSION: MI diagnoses in both the Norwegian Myocardial Infarction Register and the Norwegian Patient Register were highly correct and complete, and each of the registers could be considered as endpoint sources for the Tromsø Study. A combination of the two national registers seemed, however, to represent the most comprehensive data source overall. The benefits of using data from national registers as endpoints in epidemiological studies include faster, less resource-intensive access to nationwide data and considerably lower loss to follow-up, compared to manual data collection in a limited geographical area.

Introduction

The Tromsø Study is a population-based, prospective study consisting of seven surveys (Tromsø 1–7) conducted in the municipality of Tromsø during the period 1974–2016.1,2 The participation rate was 74–79% for the Tromsø Study 1 through 5, declining to 65–66% for the Tromsø Study 6 and 7. The study was originally established with a primary aim to study causes of cardiovascular mortality and to contribute to the prevention of cardiovascular diseases. To follow up the study participants, the Tromsø Study Cardiovascular Disease Register has been established containing information on incident fatal and non-fatal cases of cardiovascular diseases (myocardial infarction, stroke, atrial fibrillation and venous thromboembolism).3 Adjudication of cases was performed by trained personnel through review of medical records at the University Hospital of North Norway and through linkage to the Norwegian Cause of Death Registry.4

Ascertaining cases through expert review of medical records is considered the gold standard of data collection methods and is widely used in health register validation studies.5 Consequently, it is to be expected that the Tromsø Study Cardiovascular Disease Register is highly correct and complete. However, manual data collection is quite resource intensive as it involves a meticulous and time-consuming effort by trained reviewers. Given that the Tromsø Study is a prospective, ongoing study with no defined end-date, ascertainment of endpoints will be necessary for years, or rather decades, to come. In Norway, the Norwegian Patient Register was established with person identifiable information in 2008 and the Norwegian Myocardial Infarction Register followed in 2012. Thus, an opportunity emerged to investigate whether linkage to any of the national registries could replace today’s manual data collection method. In the present paper, we compare the correctness and completeness of hospitalized MI cases in the Norwegian Myocardial Infarction Register and the Norwegian Patient Register, using the Tromsø Study Cardiovascular Disease Register as gold standard.

Materials and Methods

The Tromsø Study Cardiovascular Disease Register

Fatal and non-fatal incident MI cases among Tromsø Study participants are included in the Tromsø Study Cardiovascular Disease Register. The Tromsø Study participants were linked to the Norwegian Cause of Death Registry and to the discharge diagnosis registry at the University Hospital of North Norway, which is the only hospital in the municipality of Tromsø.

To ascertain MI cases, an endpoint committee consisting of experienced physicians reviewed all hospital medical records with an International Classification of Diseases 10th revision (ICD-10) discharge diagnosis of I20-I25, I46-I48, I50, R96, R98 or R99. They also performed manual and/or electronic text searches in paper (used until 2001) and digital versions of hospital records for the term “infarction” (Norwegian or Latin equivalents) in participants with an ICD-10 discharge diagnosis of I60-I69, G45, G46 or G81. Modified World Health Organization Monitoring Trends and Determinants in Cardiovascular Disease (MONICA)/MONICA Risk, Genetics, Archiving and Monograph (MORGAM)6 criteria were used and included clinical symptoms and signs, findings in electrocardiograms, values of cardiac biomarkers, and autopsy reports when applicable.7 Study participants who moved out of the municipality of Tromsø were lost to follow-up for non-fatal events. The Tromsø Study Cardiovascular Disease Register contains endpoints from 1968 and onwards; however, the register was several years behind in data collection due to its resource-intensive data collection method and case ascertainment was not complete beyond the year 2014 at the time of the present study. The register contains dates of hospitalization, symptom onset and death, symptoms, electrocardiograms, laboratory results, percutaneous coronary intervention, findings from autopsy, whether the patient was hospitalized, and date and source of review and registration.

The Norwegian Myocardial Infarction Register

The Norwegian Myocardial Infarction Register is a national medical quality register established in 2012. According to the Norwegian Health Register Act,8 all Norwegian hospitals are obliged to register patients hospitalized with an acute MI in the Norwegian Myocardial Infarction Register, without requesting patient consent. The inclusion criteria are all patients with an ICD-10 diagnosis of I21 or I22 who were hospitalized ≤28 days after symptom onset. Acute myocardial infarctions are classified in the register according to subtypes 1, 2, 3, 4 a-c and 5.9 The register contains person identifiable information on the dates and times of symptom onset, hospital admission and discharge, as well as risk factors, medical history, symptoms and clinical findings, electrocardiographic (ECG) and echocardiographic results, plasma levels of cardiac troponins, and the use of drugs.10 Data are manually entered into the register by use of a web-based form by nurses and physicians trained in heart medicine.

The Norwegian Patient Register

The Norwegian Patient Register is an administrative, national health register covering all hospital activity within somatic and psychiatric care. The register contains person identifiable information on all hospitalizations and outpatient visits in all public hospitals and in private hospitals included in the public reimbursement policy in Norway since 2008. The register is used as a basis for reimbursement to hospitals, hospital activity statistics, waiting list statistics and for research. The Norwegian Patient Register contains demographic, administrative and health-related data, such as dates of admission and discharge, and the main and up to 20 secondary discharge diagnoses according to the ICD-10 and codes for diagnostic and therapeutic procedures, as provided by the attending physician. Data are extracted from the hospitals’ patient administrative systems based on a predefined set of rules, and cumulative data are transferred to the register on a monthly basis.

Combining the Two National Registers

The Norwegian Myocardial Infarction Register is a medical quality register and the Norwegian Patient Register is an administrative discharge register. Hence, they differ in terms of data collection methods, inclusion criteria and contents. These differences are likely to reflect specific data quality issues in the two registers. In Norway, the National Institute of Public Health yearly links these two registers in order to assess the registers’ coverage, assuming that a combination of the two yields the most precise estimate of hospitalized MI cases available through registers. For the same reason, we chose to validate a combination of the two national registers in addition to validating the registers separately.

Study Population

Based on all participants ≥18 years of age in the Tromsø Studies 1 through 6 from 1974 to 2008 (N=39,866), we defined our study population as individuals alive and residing in the municipality of Tromsø by 1.1.2013 (N=23,665) (Figure 1).

Figure 1

Study population. Participants in the Tromsø Study 1–6.

Statistical Analysis

Data from the three different registers were linked using the unique national identification number issued by the National Population Register to all residents of Norway.

We defined the gold standard as follows: All incident, hospitalized MI cases occurring in 2013–2014 classified as definite or probable MI in the Tromsø Study Cardiovascular Disease Register. Cases classified as possible MI were included if they were also present in one of the national registers. To enable comparison with the Norwegian Myocardial Infarction Register and the Norwegian Patient Register, we excluded non-hospitalized fatal and non-fatal (N=3) cases.

In the Norwegian Myocardial Infarction Register, we defined incident MI cases as the first hospitalization with an MI diagnosis in 2013–2014 among patients who had participated in the Tromsø Study at least once and were still living in the municipality. Cases registered as recurrent MI were excluded.

We defined incident MI cases in the Norwegian Patient Register as the first hospitalization during 2013–2014 with a main or secondary diagnosis of MI (ICD-10 codes I21 or I22). Only patients who had participated in the Tromsø Study at least once and still living in the municipality of Tromsø were included. Patients registered with an MI diagnosis in the period 2008–2012 were excluded.

Additionally, we analyzed a combination of the Norwegian Myocardial Infarction Register and the Norwegian Patient Register, defined as cases present in either of the registers.

Based on the established gold standard, we classified cases as true positives (TP), true negatives (TN), false positives (FP) or false negatives (FN). We defined data completeness as equivalent to the sensitivity (TP/(TP+FN)), ie the proportion of cases of true MI according to the gold standard that were also present in the registers. We defined data correctness as equivalent to the positive predictive value (PPV) (TP/(TP+FP)), ie the proportion of MI cases present in the registers that were cases of true MI according to the gold standard.11 The 95% confidence intervals (CI) were calculated using the Clopper-Pearson Exact method.

This study was assessed by The Regional Committee for Medical and Health Research Ethics (REK) and was exempted from requiring ethical approval. All Tromsø Study participants have signed a written, informed consent, and the study was approved by the Data Protection Officer at UiT, The Arctic University of Norway. All statistical analyses were performed using IBM SPSS Statistics version 26.

Sub-Analyses

Following the initial validation analyses, one of the authors (A.N.) performed a post-hoc review of hospital medical records for all cases classified as false negative or false positive to outline possible explanations for the discrepancies between the registers. The author used the same strict adjudication methods as used in the initial case ascertainment for the Tromsø Study Cardiovascular Disease Register, classifying MI cases based on the MORGAM criteria (described above).

Furthermore, since the Tromsø Study Cardiovascular Disease Register does not have access to non-fatal cases occurring outside the study area, Tromsø Study participants have been considered lost to follow-up for non-fatal events if they moved out of the Tromsø municipality. Consequently, it was of interest to investigate the magnitude of the population mobility and its effect on the completeness of MI cases in the Tromsø Study Cardiovascular Disease Register. We conducted a simple sub-analysis of participants who were alive by 1.1.2013 but had moved from the municipality of Tromsø (n=9577).

Results

We identified 153 MI cases in the gold standard, compared to 136 and 139 cases in the Norwegian Myocardial Infarction Register and the Norwegian Patient Register, respectively. The combination of the two national registers identified 145 incident MI cases. Table 1 describes the distribution of true and false positives and negatives in the national registers compared to the gold standard. Estimated measures of correctness and completeness of the Norwegian Myocardial Infarction Register indicated a PPV of 97.1% (95% CI 92.8–99.2%) and sensitivity of 88.2% (95% CI 82.0–92.9%) (Figure 2). The post-hoc review of medical records suggested that among the 18 cases classified as false negative, four cases were actually true negative (Table 2). The four false positive cases were elderly (age 85–95) females with type 2 infarction who should have been registered in the gold standard and were thus actually true positive. By using the revised gold standard, the Norwegian Myocardial Infarction Register had a sensitivity of 90.8%, and a PPV of 100%.

Table 1

Distribution of True and False Positives and Negatives in the National Registers Compared to the Gold Standard (The Tromsø Study Cardiovascular Disease Register)

	Gold Standard: The Tromsø Study Cardiovascular Disease Register
	MIa	No MI	Total
The Norwegian Myocardial Infarction Register
MI	135	4	139
No MI	18	23,508	23,526
Total	153	23,512	23,665
The Norwegian Patient Register
MI	131	5	136
No MI	22	23,507	23,529
Total	153	23,512	23,665
Combination of the Norwegian Myocardial Infarction Register and the Norwegian Patient Register
MI	140	5	145
No MI	13	23,507	23,520
Total	153	23,512	23,665

Abbreviation: aMI, myocardial infarction.

Figure 2

Estimated positive predictive value (PPV) and sensitivity for myocardial infarction diagnoses in the national registers compared to the gold standard (the Tromsø Study Cardiovascular Disease Register). Error bars show 95% confidence intervals.

Table 2

Results from Post-Hoc Review of Medical Records for All False Negative and False Positive Cases Identified in the Validation Analyses

	False Negatives		False Positives
	NorMIa	NPRb	NorMIa		NPRb
Incorrect Registration in Gold Standardc
Did not fulfill myocardial infarction (MI) criteria	3	3
In-hospital cardiac arrest, no MI (ICD-10 code I46)	1	1
Type 2-infarction			4		4
Total	4	4	4		4
Correct Registration in Gold Standardc
No MI diagnosis in hospital records, but fulfilled MI criteria	6	6
MI diagnosis in hospital records and fulfilled MI criteria	7	11
Patient treated for MI abroad	1	1
Did not fulfill myocardial infarction (MI) criteria					1
Total	14	18
Total	18	22	4		5

Notes: aThe Norwegian Myocardial Infarction Register (NorMI). bThe Norwegian Patient Register (NPR). cThe Tromsø Study Cardiovascular Disease Register.

In the Norwegian Patient Register, estimated PPV of incident MI diagnoses was 96.3% (95% CI 91.6–98.8%) and sensitivity was 85.6% (95% CI 79.0–90.8%). The five false positive cases included the same four false positives found in the Norwegian Myocardial Infarction Register. Among the 22 false negative cases, the post-hoc review of hospital medical records revealed that six of the cases did not have an MI diagnosis in the hospital medical records, yet they fulfilled the MI criteria. By using the revised gold standard, the Norwegian Patient Register had a sensitivity of 88.2%, and a PPV of 100%.

Combining the Myocardial Infarction Register and the Norwegian Patient Register, estimated PPV was 96.6% (95% CI 92.1–98.9%) and sensitivity was 91.5% (95% CI 85.9–95.4%). This analysis resulted in the same five false positive cases as in The Norwegian Patient Register and reduced the false negative cases to only 13. By using the revised gold standard, the combination of the two national registers had a sensitivity of 94.1%, and a PPV of 100%.

Among the 9577 participants who were alive 1.1.2013 but had moved out of the municipality of Tromsø, we identified 65 and 67 incident MI cases in the Norwegian Myocardial Infarction Register and the Norwegian Patient Register, respectively. Only 11 of these cases were registered in the Tromsø Study Cardiovascular Disease Register, thus indicating that the register was missing endpoints for 83.5% of the participants who had moved (data not shown).

Discussion

The aim of this study was to investigate whether linkage with national registers can replace manual endpoint data collection methods in an epidemiological study.

MI diagnoses in both the Norwegian Myocardial Infarction Register and the Norwegian Patient Register were highly correct and complete. Previous studies have pointed out that manual data collection by trained personnel, as in the Norwegian Myocardial Infarction Register, can be beneficial to the PPV as it minimizes the risk of entering false positive cases into the register, while at the same time completeness of cases may be hindered due to the resource-intensive nature of this data collection method.10,12 A Danish study emphasized the benefits of combining national registers to achieve a comprehensive representation of cases, as different registers have distinct data quality strengths and weaknesses.13 Based on the same rationale, we analyzed a combination of all MI cases in the Norwegian Myocardial Infarction Register and main and secondary diagnoses of MI in the Norwegian Patient Register. The results indicated an excellent degree of correctness and completeness, and this combination seemed to represent the most comprehensive data source overall.

The post-hoc review of medical records revealed that four of the five false positive cases actually met the MI criteria and were thus missing in the Tromsø Study Cardiovascular Disease Register; all cases concerned elderly females (age 85–95) hospitalized with type 2 infarction as secondary diagnoses. Furthermore, four of the false negative cases did not fulfill the MORGAM criteria for MI, yet they were registered with MI in the Tromsø Study Cardiovascular Disease Register. Taking this into account, the correctness and completeness of the national registers is somewhat underestimated in our results. This illustrates that it is important for users of health register data to be aware that all registers, even those based on meticulous manual data collection methods, contain erroneous registrations, and that 100% correctness and completeness is virtually unattainable. However, the utility and benefits of large health registers are not impeded by a slightly imperfect correctness and completeness.

Interestingly, the post-hoc review of medical records also found that several cases lacked an MI diagnosis in the hospital records but fulfilled the MORGAM criteria for MI. This finding underscores the well-known challenges in diagnosing MI correctlyfor instance, when a rise and fall of troponin occurs in an atypical clinical setting with no clear evidence of ischemia. In our study, these cases were defined as false negatives due to our choice of gold standard; however, in complex cases, it is possible that the attending physician’s assessment was more correct than the review of medical records performed years after the incident. Consequently, there is some uncertainty regarding whether these cases were “true” false negatives.

We found that the population mobility in Tromsø was considerable, approximately 25% of the Tromsø Study participants had moved out of the municipality before 1.1.2013, suggesting that the Tromsø Study regularly misses endpoints from a considerable proportion of their participants. Although 11 MI cases among those who moved out were present in the Tromsø Study Cardiovascular Disease Register because they had been treated at the University Hospital of North Norway, an additional 54 and 56 incident MI cases in 2013–14 were detected in the Norwegian Myocardial Infarction Register and the Norwegian Patient Register, respectively. Further research is necessary to investigate potential endpoint bias in the Tromsø Study; however, the impact of population mobility on loss to follow-up should be of concern to any population-based study.

In conclusion, our results indicate that data from national registers is highly correct but slightly less complete than data collected by manual review of medical records. However, considering the issues of loss to follow-up due to population mobility and the delay in data collection in the Tromsø Study Cardiovascular Disease Register, collecting data from national registers would arguably lead to more complete and timely endpoints for the Tromsø Study. Our results are generalizable to other population-based, epidemiological studies that collect MI endpoints manually, provided there is access to validated, high-quality regional or national registers.

Validation studies of various health registers and administrative databases have been carried out over the years with inconsistent results. The inconsistencies may be due to true differences in validity or to methodological issues like sampling methods, sample size and definitions of gold standards. Results from previous studies and systematic reviews indicate PPV and sensitivity ranging from less than 50% to over 90%.5,10,14–17 In line with our study, most studies found that the PPV was higher than the sensitivity. Of particular interest, a previous study validating MI diagnoses in the same two registers as our study (The Norwegian Myocardial Infarction Register and the Norwegian Patient Register) found similar results as we did, with a sensitivity of 85–86% and a PPV of 95–97%.10 Previous studies have suggested that in-hospital mortality is higher among cases missed in the registers,12,18 and users of register data should be aware of this potential bias. In our study, we did not find any signs of selection bias; however, the study population was too small to draw conclusions in any direction.

In this study, we had access to person identifiable data sets from three health registers, thus allowing for identification and linkage of each unique MI case across the data sets. Another strength of the study is the comparisons between different types of registers, which highlights the importance of data quality awareness when using data from different types of health registers. The main limitation of this study was the inability to unambiguously identify incident MI cases in the two national registers. The Norwegian Patient Register collects all hospitalized MI cases and does not include information on incident versus recurrent cases. To exclude recurrent cases, we relied on a search for previous MI diagnoses in the period 2008–2012; however, this method introduces some uncertainty to the identification of incident cases. The Norwegian Myocardial Infarction Register, on the other hand, contains a specific variable for previous MI. We used the information in this variable to exclude recurrent MI cases, but this method also carries some uncertainty.

Importantly, our study only investigated the hospitalized MI cases, as the national registers exclude non-hospitalized cases. In the Tromsø Study Cardiovascular Disease Register, we identified three non-hospitalized MI cases in 2013–2014. In the event of conversion from manual data collection to linkage with national registers, fatal non-hospitalized cases can be collected from the Norwegian Cause of Death Registry. The impact of missing the rare non-fatal non-hospitalized cases of MI will be negligible.

Conclusion

We found that MI diagnoses from national registers had acceptable levels of correctness and completeness to be considered as endpoint sources for a population-based epidemiological study. A combination of the Norwegian Myocardial Infarction Register and the Norwegian Patient Register indicated excellent correctness and completeness and was considered the overall best data source. The benefits of using data from national registers as endpoints in epidemiological studies include faster, less resource-intensive access to nationwide data and considerably lower loss to follow-up, compared to manual data collection in a limited geographical area.