Completeness in the Swedish Fracture Register and the Swedish National Patient Register: An Assessment of Humeral Fracture Registrations.

PURPOSE: Register-based clinical research is important. However, it is essential that the collected data are reliable for the registers to be a valuable source of information. This study evaluated the quality of humeral fracture data in the Swedish Fracture Register (SFR) and in the Swedish National Patient Register (NPR). Furthermore, a model for improved case ascertainment was developed for future validation processes.
MATERIALS AND METHODS: Data were obtained from the NPR and SFR for all individuals aged ≥16 years with an acute humeral fracture ICD-code treated at Sahlgrenska University Hospital. The true number of humeral fractures ("gold standard") was determined by cross-linkage between the two registers and a medical charts review. The completeness of registrations in each register was measured as the proportion of registrations compared with the gold standard, and accuracy was measured as positive predictive values (PPV).
RESULTS: The NPR demonstrated a high level of completeness (97%) and lower accuracy (PPV 70%) for acute humeral fractures, whereas the SFR had slightly lower completeness (88%) but perfect accuracy (PPV 100%). The most common systematic error was the registration of re-admissions as acute fractures in the NPR (84% of all erroneous registrations). With this knowledge, an adjustment model for NPR data was constructed to increase the accuracy of fracture registrations (PPV 92%) without excluding valid registrations.
CONCLUSION: Data from the NPR tend to overestimate the true number of fractures, and proper case selection is needed in order for the data to function as a solid basis for epidemiological research and healthcare planning. In contrast, the SFR constitutes a complete, accurate and efficient source of information.

Introduction

Observational register-based studies are able to address clinical issues that cannot easily be resolved by prospective randomized, controlled trials.1 Sweden, along with the other Nordic countries, has a long tradition of register-based clinical research.2 The Swedish Board of Health and Welfare administers several nationwide population-based databases, such as the National Patient Register (NPR).3 The NPR is an administrative database which collects data on all diagnoses and surgical procedures in in-patient and out-patient specialist care. Alongside the NPR, the medical profession has developed more than 100 disease-specific national quality registers (NQRs), which have contributed vital knowledge to improve health care.2 The world’s only NQR for fractures, the Swedish Fracture Register (SFR), has more than 500,000 fractures registered and is used continuously to analyze trends in treatment, adherence to clinical guidelines and treatment outcome.4–6 In recent years, administrative databases of electronic health records and professional-run quality registers have gained increasing popularity among clinical epidemiologists and serve as important tools in surveillance of diseases and injuries.7

The continuous assessment of data quality is important for registers to be effective and useful. Different areas, including accuracy, coverage and completeness, need to be validated. Several methods have been used to evaluate completeness in the NQRs. The most common is cross-linkage with another national register.2 This method requires a valid reference, a “gold standard”, with which the quality register of interest should be compared. Almost half the Swedish NQRs have used the NPR as a reference.8 However, administrative databases such as the NPR may be subject to diagnostic and coding errors.9 An imperfect “gold standard” generates a faulty validation. Only a few previous studies have evaluated the completeness of the NPR, all with conflicting results with regard to the total number of patients.9–12 No previous study has examined the completeness of the Swedish NPR for fractures, nor has the completeness of the SFR been analyzed in detail.

The aim of the present study was to evaluate the completeness and accuracy of humeral fracture registrations in the SFR and in the NPR regarding fractures treated at Sahlgrenska University Hospital (SUH). Furthermore, the study aimed to identify systematic errors in the registers to suggest ways to increase future validity.

Materials and Methods

Study Design and Setting

The study was designed as a regional prevalence study using data from two independent registers, the NPR and the SFR. The NPR automatically collects longitudinal data and reached nationwide coverage for in-patient admissions in 1987 and for out-patient specialist visits in 2001.3 The NPR is administered by the Swedish Board of Health and Welfare. Data are continuously delivered from patient-administrative systems and medical records for in-patient care in hospitals and out-patient departments in hospitals. It is mandatory by law (SFS 1998:543 6§) for all healthcare providers in Sweden to submit data to the NPR. The information delivered to the NPR includes personal identity number (PIN), gender, name of hospital/out-patient department, admission and discharge dates, as well as diagnostic coding according to the Swedish version of the international classification of disease (ICD)-10 system.9 Regarding fractures, neither the time of injury nor the laterality of the injured body part is recorded. As a result, the NPR is unable to distinguish a readmission due to causes related to a previous fracture from an admission due to a new fracture, if an acute fracture code is used repeatedly.

The SFR is a web-based NQR created by orthopedic surgeons and data entry is performed by the attending physician.4 Detailed data on patient characteristics, date and mechanism of injury, fracture type, treatment modality, level of training of the treating physician and patient-reported outcome measures (PROMs) are collected. Data are entered sequentially, meaning that altered treatments, reoperations or later treatments by another healthcare provider are connected to the initial fracture registration. Fractures sustained abroad and fractures in persons without a Swedish PIN are not included. Participation in the SFR is voluntary and it is not necessary for the healthcare provider to register data in the register. The coverage of the SFR has rapidly increased in recent years and in 2021, full national coverage was reached. The creation, implementation, validation and the registration process of the SFR have previously been described in detail.13–15

Methods

One quantitative method of assessing completeness is independent case ascertainment, which means that independent data sources are compared with each other in order to find true cases.16,17 While the NPR uses several sources for case notification (including data from public and private caregivers), the SFR is an independent source that is not used for case notification by the NPR. As a result, data can be compared between these registers in order to find ascertained cases and additional cases not primarily registered in the respective register.

In the present study, data were extracted from the NPR and the SFR on all individuals aged ≥ 16 years, with an acute humeral fracture diagnostic code (ICD 10 S42.2/S42.3/S42.4), treated at SUH between 1 January 2011 and 31 December 2012 (Figure 1). By using Swedish personal identification numbers (PIN), the cross-linkage of individuals was performed in the NPR and the SFR. An event (fracture) recorded in both registers was considered a valid fracture registration (matching registration). A fracture only recorded in one register was regarded as a mismatching registration. In patients with mismatching registrations, the medical records were reviewed and reasons for the missing registration were searched for. A first time visit for a humeral fracture primarily treated at SUH during the study period was referred to as an “actual fracture”, whilst all other reasons for the registration (ie re-admissions and non-humeral fractures registered as acute humeral fractures) were referred to as “erroneous registrations” (Table 1). The “true” number of humeral fractures during the study period was defined by (a) all fractures registered in both registers and (b) all unique fractures registered in either register and verified by the medical records. This “gold standard” was then used to assess completeness in both the NPR and the SFR.

Figure 1

Flowchart of the study design and the assessment of the “true” number of acute humeral fractures at Sahlgrenska University Hospital in 2011–2012. The “true” number of fractures was defined by (a) all the fractures registered in both registers and (b) all the unique fractures registered in either register and verified by the medical records.

Table 1

Reasons for Registration in the National Patient Register

			Number	% of All Registrations (n. 2173)	% of Adjusted Registrations* (n. 1648)
Humeral fracture registration, actual fracture			1513	69.6	91.8
Re-admission	Re-registration at SUH during study period		317	14.6	Excluded
	Humeral fracture prior to study period	≤ 2 years prior to re-admission	186	8.6	Excluded
		≥ 2 years prior to re-admission	16	0.7	1.0
	Humeral fracture primarily treated at another Swedish orthopedic department		22	1.0	Excluded
	Humeral fracture sustained and primarily treated abroad		11	0.5	0.7
Non-humeral fracture registration			108	5.0	6.6
Total			2173

Note: *Adjustment algorithm used on NPR data for routine completeness calculation in the Swedish Fracture register.

Abbreviation: SUH, Sahlgrenska University Hospital.

Statistics

Completeness was calculated for both the SFR and the NPR as the total number of registrations in respective register divided by the “true” number of cases. To quantify accuracy, the positive predictive values (PPV) of the two registers were calculated. The PPV was calculated as the proportion of patients registered as having an acute humeral fracture who truly had an acute humeral fracture with a first-time visit for that fracture at SUH in 2011–2012. Completeness was assessed descriptively and the PPV was calculated with 95% confidence intervals (CI) according to normal approximation.

Adjustment of NQR Data

Nonmatching registrations were analyzed to identify systematic errors. To further sharpen accuracy regarding acute humeral fracture registration in the NPR, an algorithm was designed to exclude as many nonmatching registrations as possible in the NPR data, without excluding valid registrations of humeral fractures. The algorithm was then applied to data from 2016–2018 from other parts of Sweden (all counties attached to the SFR at least one year prior to 2016). By 2015, approximately 40% of the departments treating orthopedic trauma in the country participated in the SFR.

The study was approved by the Central Ethical Review Board, Gothenburg (Ref no. 1018–13) and all data accessed complied with relevant data protection and privacy regulations. All patients were informed of the registration in the Swedish Fracture Register and that they had the right to decline registration. According to Swedish legislation (SFS 1998:543 6§), national quality registers do not require signed consent from the individual registered patients.

Results

During the study period, 1876 patients with 2197 acute humeral fractures were reported to the NPR from SUH and 1375 patients with 1375 acute humeral fractures were registered in the SFR from SUH. Twenty-four patients without a Swedish PIN were excluded from the NPR data, leaving 2173 registrations in the NPR eligible for analysis (Figure 1). There were 1324 matching registrations (true cases). A total of 900 nonmatching registrations were identified, of which 51 were in the SFR and 849 in the NPR. The 51 nonmatching registrations in the SFR were all actual humeral fractures missing in the NPR. Of the 849 nonmatching registrations present in the NPR, 189 were actual fractures that had not been registered in the SFR. As a result, the estimated number of true humeral fractures treated at SUH during the study period was 1564.

Of the 2173 registrations in the NPR, 1513 (70%) were actual humeral fractures treated at SUH in 2011–2012 (Table 1). The remaining 30% (n=660) were erroneous registrations (ie registrations of re-admissions or non-humeral fracture registrations). The majority of erroneous registrations were related to re-admissions for a previously registered humeral fracture (n=552; 84%), but 108 were registrations of other medical conditions (fractures in other parts of the body; n=58, other humeral injuries but no fracture; n=33, and other various conditions; n=17). Of the re-admissions, more than half (n=317; 57%) were additional registrations for a humeral fracture already registered during the study period and the remaining registrations (n=235; 43%) were re-admissions for an acute humeral fracture either sustained prior to the study period or primarily treated elsewhere. There were no patients with bilateral fractures or two different humeral fractures in the same humeral segment.

The NPR had a completeness of 97% and a positive predictive value (PPV) of 70% (95% CI: 68–72) for acute humeral fractures (Tables 2 and 3). When all the registrations from the NPR were used as a reference for completeness calculation, the SFR had an apparent completeness of 62%. However, when erroneous registrations were excluded from the NPR data and true fractures were used as a reference, the actual completeness in the SFR was 88%. The PPV in the SFR for acute humeral fractures was 100% (95% CI: 100–100) (Tables 2 and 3).

Table 2

Calculated Completeness with Different Reference Data from the National Patient Register (NPR)

	Reference Data from the NPR	Number of Registrations	True Fractures Missing	Formula for Completeness	Completeness (%)
NPR	All registrations	2173	51	2173/(1324+849+51)	97.7
	True number of humeral fractures	1513	51	1513/(1324+189+51)	96.7
	Adjusted registrations in the NPR*	1648	51	1648/(1324+324+51)	97.0
SFR	All registrations	1375	189	1375/(1324+51+849)	61.8
	True number of humeral fractures	1375	189	1375/(1324+51+189)	87.9
	Adjusted registrations in the NPR*	1375	189	1375/(1324+51+324)	80.9

Note: *Adjustment algorithm used on NPR data for routine completeness calculations in the SFR.

Abbreviation: SFR, Swedish Fracture Register.

Table 3

Positive Predictive Value and Relationship to the True Number of Fractures (N. 1564) with Different Reference Data in the National Patient Register (NPR)

	Reference Data from the NPR	Number of Registrations	Number of Actual Fractures	Erroneous Registrations	Positive Predictive Value (PPV) %	Confidence Interval (95%) of PPV	Relationship to the True Number of Fractures (%)
NPR	All registrations	2173	1513	660	69.6	67.7–71.6	138.9
	True number humeral fractures	1513	1513	0	100	100–100	96.7
	Number of patients (unique PINs)	1852	1513	339	81.7	79.9–83.5	118.4
	Adjusted registrations in the NPR*	1648	1513	135	91.8	90.5–93.1	105.4
SFR	All registrations	1375	1375	0	100	100–100	87.9

Note: *Adjustment algorithm used on NPR data for routine completeness calculations in the Swedish Fracture register.

Abbreviation: PIN, personal identification number.

Information on erroneous registrations in the NPR is shown in Table 1. Based on these findings, an adjustment algorithm for NPR data was constructed accordingly; if all the individuals with an acute humeral fracture code (ICD-10 S42.2–4) in the NPR that had been registered with the same fracture code within two previous years were excluded, the accuracy for actual acute humeral fractures in the NPR improved. With this modified NPR data, the PPV in the NPR increased to 92% (95% CI: 90–93) without the loss of actual acute fracture registrations (Table 3).

When NPR data extracted via the adjustment algorithm were used as reference data (“gold standard”), the completeness for humeral fractures in the SFR in 2016–2018 was calculated to be between 75–90% in different counties (Figure 2).

Figure 2

Calculation of completeness in the SFR (Swedish Fracture Register; attached counties) and NPR (National Patient Register) in 2016–2018 according to the adjustment algorithm.

Discussion

This study demonstrates that the NPR has a high level of completeness for actual acute humeral fractures, 97%. However, a large proportion of registrations in the NPR were due to re-admissions or non-humeral fractures, and the PPV was as low as 70%. This finding gives valuable information on how to interpret data from the NPR and avoid bias in epidemiological studies. The SFR at Sahlgrenska University Hospital showed a lower completeness level than the NPR (88%) but a perfect PPV (100%) for acute humeral fractures. This demonstrates that it is possible to achieve valid registrations in a NQR based on voluntary means.

A high degree of validity is of significant importance whenever register-based, observational data are used for research purposes or healthcare planning.1,9 In the Swedish NPR, the PPV has been reported to be between 85 and 95% and the completeness between 50 and 98% for various clinical conditions.9,12,18–20 Similar rates have been reported from disease-specific administrative databases in other parts of the world, however validation data regarding fracture registrations from outside of the Nordic region is sparse.21–24 The Nordic countries have a long tradition of register-based research and many validation studies originate from these countries. The completeness of 97% for acute humeral fractures in the NPR reported in this study is comparable with hip-fracture data in the NPRs of other Nordic countries. Lofthus reported a completeness rate of 89% for hip-fracture registrations in the Norwegian NPR from four centers in Oslo, while Emaus reported a 100% completeness rate from northern Norway, when compared with medical records.11,25 In Finland, Sund reported a completeness rate for hip-fracture registrations of 97% in their NPR compared with audited data from a single center.26 To the best of our knowledge, no previous validation study of non-hip fractures has been performed on a Nordic NPR.

Using the estimated number of “true” humeral fractures in the current study, 3.3% of these fractures were missing in the NPR. In spite of this, the NPR reported 39% more humeral fractures than the “true” number. One reason for this overestimation of cases is the lack of injury dates in the NPR. Re-admissions and follow-up visits for one fracture may generate several acute fracture codes in the medical charts and thereby multiple registrations in the NPR.27 A Norwegian study indicated that the Norwegian NPR overestimated the total number of hip fractures by 19% due to the inclusion of re-admissions for the same hip fracture.11 This phenomenon probably applies to other medical conditions, as approximately 30% of patients with a stroke in the NPR turned out to have had an earlier stroke when medical charts were reviewed.9

Of the humeral fractures registered in the NPR, only 70% were registrations of an actual acute humeral fracture. One quarter (25%) of all registrations comprised re-admissions or follow-up visits for a previous fracture. Pure diagnostic coding errors were less common, but still 5.0% of the humeral fracture registrations in the NPR were unrelated to a humeral fracture diagnosis. This aligns with the registration of other medical conditions in the NPR (dementia 10% and stroke 6%).12,18 In comparison, the registrations in the SFR were all acute and unique humeral fractures. This demonstrates a high level of accuracy in the registrations in the SFR, probably due to the active registration process by orthopedic surgeons. Together with a completeness of 88% for acute humeral fractures, it indicates that data from the SFR collected at SUH are valid for epidemiologic studies.5,28

According to the National Board of Health and Welfare in Sweden, as many as 39 NQRs regularly use the NPR as a reference when assessing completeness.8 Completeness rates for the NQRs are reported to be between 6–97% with this method of validation. In the current study, an apparent completeness of 62% for the SFR increased to 88% when the NPR was corrected for aberrant data. This highlights the necessity for all NQRs to validate the quality of their reference data, ie the NPR, in order to present trustworthy completeness rates. The validation of the reference data (ie the NPR) in the current study was conducted by independent case ascertainment enabled by PINs and medical records reviewing for non-matching registrations. This method is well established for validation studies but less useful for routine completeness calculations in NQRs, since it is time consuming and requires the handling of PINs.16,17 Routine transfer of information between registers is not allowed by Swedish legislation and certain rules apply to identifiable data (through the PINs, for instance) for integrity protection purposes. For this reason, the extraction and handling of PINs always require separate ethical approval.2,29

A surprisingly large discrepancy between the data reported to the NPR and the medical records indicates that an adjustment algorithm for humeral fracture NPR data could be warranted. In particular, if routine completeness calculations of the SFR used the NPR as a reference, such an adjustment algorithm would significantly increase the quality of the validation process.18,27 Similar algorithms for enhancing the case selection in the NPR have previously been constructed for the monitoring of stroke and myocardial infarction.18,30 The constructed adjustment algorithm is simple and will hopefully be useful in future validation work. When applying the constructed adjustment algorithm to the NPR data in this study, all the re-occurring registrations (n=317/317; 100%), the majority of humeral fractures sustained prior to the study period (n=186/202; 92.1%) and all humeral fractures primarily treated in other departments in Sweden (n=22/22; 100%) were correctly excluded. However, erroneous non-humeral fracture registrations (n=108) and fractures sustained and primarily treated abroad (n=11) were not excluded. Unfortunately, the algorithm also excluded new humeral fractures (same humeral location, ie ICD10 S42.2–4) sustained within two years of the first fracture and all contralateral fractures, since laterality is not registered in the NPR. However, this appears to be uncommon for humeral fractures, since no re-occurring or bilateral fracture was found in this cohort.5 When the adjustment algorithm was used on the NPR data, the PPV increased from 70% to 92%, indicating that such refined NPR data could be valuable in future SFR completeness calculations. There is reason to believe that the adjustment algorithm constructed in this study is warranted for NPR data in all parts of Sweden, since no substantial difference in the reporting of diagnoses have been demonstrated between different care givers obliged to report to the NPR.9 However, the adjustment algorithm needs further validation for different hospitals and counties in Sweden.

When using adjusted NPR data as a reference, the completeness for humeral fracture registration in the SFR was between 75–90% for various counties. This is in agreement with the reported completeness rates of the diagnose-specific hip fracture registers in the Nordic countries, but lower than the Swedish hip and knee arthroplasty registers (completeness rate 98% and 97% respectively, for primary procedures in the arthroplasty registers).31–34 Still, the completeness in the SFR is surprisingly high considering the different register set-ups; the arthroplasty registers have personnel employed for registration, as well as motivated and only surgically treated patients, and the SFR where data are collected and filed by the attending orthopedic physician. Registration in the SFR is non-compulsory and often performed in an acute setting at the accident and emergency department, without additional time reserved for the procedure. Despite these inherent difficulties, it appears to be possible to obtain high completeness rates for fracture registration in the SFR. In fact, the true completeness in the SFR might be even higher than the numbers presented in this study, since the reference data of the NPR still overestimates the number of acute humeral fractures in this material.

Limitations

One limitation with regard to generalizability is the confinement to a single center. However, this enabled a thorough medical chart review. A possible source of error are fractures that were not registered in either the NPR or the SFR. These cases would not be ascertained by the methodology used in this study. However, since the SFR and NPR demonstrated a high degree of matching registrations, this scenario seems unlikely according to a capture-recapture methodology.35,36

Conclusion

The first validation study of the NPR and the SFR regarding the completeness and accuracy of humeral fracture registrations indicates a high level of completeness for humeral fractures in the NPR, but NPR data tend to grossly overestimate the number of fractures. For this reason, unadjusted NPR data appear to be unsuitable for epidemiologic studies of fractures or as reference data in completeness calculations. On the other hand, data from the SFR demonstrated both acceptable completeness and perfect accuracy. This study illustrates that monitoring case ascertainment in new registers such as the SFR, as well as in established data sources such as the NPR, is valuable.