Completeness and positive predictive value of registration of upper limb embolectomy in the Danish National Vascular Registry.

OBJECTIVE: TO EVALUATE COMPLETENESS AND POSITIVE PREDICTIVE VALUE OF THE DANISH NATIONAL VASCULAR REGISTRY REGARDING REGISTRATION OF THE SURGICAL PROCEDURES: embolectomy of brachial, ulnar, or radial artery. STUDY DESIGN AND SETTINGS: The study was based on first-time embolectomies in the brachial, ulnar, or radial artery performed in Denmark from January 1, 1990 to December 31, 2002. The data were primarily retrieved from the Danish National Vascular Registry and secondarily from the Danish National Registry of Patients. Medical records were retrieved using a standardized form.
RESULTS: In total, 1433 incident cases of first-time embolectomy were found in both registries. The positive predictive value of the registration was 97.5% (95% confidence interval [CI]; 96.4-98.4). The degree of completeness was 86.5% (95% CI; 84.3-88.5). For the registration period from 1990 till 1996 the degree of completeness was 78.2% (95% CI; 74.4-81.7), and from 1997 till 2002 it was 93.8% (95% CI; 91.6-95.7).
CONCLUSION: The completeness and positive predictive value of registration of embolectomy in the upper limb in the Danish National Vascular Registry was 86.5% and 97.5%, respectively. This registry can be a valuable tool for epidemiological research and quality-monitoring.

Introduction

Embolectomy is an established surgical treatment of acute embolism in the upper limb. Embolism in the upper limb is a rare disease, and the true incidence is not known. The disease is mostly very painful, and it can necessitate brachial amputation. After embolism in an upper limb, reported rates of brachial amputation range from zero to 18% despite prompt surgical intervention.1

Use of data collected for quality control or for administrative reasons (secondary data) for research into rare diseases, such as upper limb embolism and its treatment, can be cost and time effective. It gives the opportunity to observe a large cohort over a long period of time.

Many databases are not established for research purposes (primary data), but for quality control or for administrative purposes. The latter should, ideally, be planned according to rigorous standards, since it can be very cost and time effective to use secondary data in research.2 If quality of data is inadequate conclusions can be biased. Use of secondary data for research purposes therefore requires a reasonable degree of accuracy and completeness.3,4

It is very convenient to use secondary data sources in Denmark because there are many population-based databases which can be linked via a unique civil registration number. Several studies have validated Danish databases. Although many of them have reported good data quality,3,5–8 data quality can differ between different diagnoses in the same database.5,7,9–12 To our knowledge there is only one study evaluating surgical procedure coding in the Danish National Vascular Registry.13 It was designed as a reproducibility study of the data regarding two surgical procedures. The authors found good reproducibility of the surgical procedure coding, but a study examining the completeness of the Danish National Vascular Registry has not yet been conducted.

The purpose of the present study, therefore, is to evaluate completeness and positive predictive value of the Danish National Vascular Registry regarding registration of the surgical procedures: embolectomy in the brachial, ulnar, or radial artery.

Methods

Study population

The study was based on first-time embolectomies in the upper limb performed in Denmark from January 1, 1990 to December 31, 2002. During the study period the total population of the country increased from about 5.1 to 5.4 millions. Data about the background population were uploaded from Statistics Denmark.14 The health care system in Denmark was, during the period studied, tax-financed, nonprofit and free of charge for all inhabitants.

Data sources

The data were primarily retrieved from the Danish National Vascular Registry and secondarily from the Danish National Registry of Patients. We used a record of first embolectomy in the brachial, ulnar, or radial artery performed in the period from 1990 to 2002 to identify our cases.15,16 Emigration and vital status can be followed in the Danish Civil Registration System using the patients’ unique civil registration number.17

The Danish National Vascular Registry

The Danish National Vascular Registry was initiated in 1989 at the University Hospital of Copenhagen, Rigshospitalet, and over the following years it was accepted by other vascular departments and became a national registry.15 The purpose of the registry is quality-monitoring of procedures. Information in the Danish National Vascular Registry includes patients’ civil registration numbers, admission to hospitals and departments, dates of admission and discharge, and surgical procedures using the Danish Classification of Surgical Procedures (until December 31, 1995) or the Nordic Classification of Surgical Procedures (from January 1, 1996) coding system. The surgical procedure codes are assigned by the operating surgeon immediately after operation.

The Danish National Registry of Patients

The Danish National Registry of Patients was established in 1977 and records 99.4% of all nonpsychiatric hospital admissions in Denmark.11,16,18 Since 1995 it has also included data on outpatient and emergency clinic patients. Reporting to the Danish National Registry of Patients is mandatory and it contains data on patients including their civil registration number, hospital and department, dates of admission and discharge, surgical procedures performed using the Danish Classification of Surgical Procedures or the Nordic Classification of Surgical Procedures coding system, and one or several discharge diagnoses using the International Classification of Diseases, 8th Revision (ICD-8) or the International Classification of Diseases, 10th Revision (ICD-10). All discharge diagnoses are assigned by the discharging physician.

The Danish Civil Registration System

All citizens in Denmark are assigned a unique 10-digit code, given to each individual having, or having had an address in Denmark since April 1968.17 Data are stored in the Danish Civil Registration System containing electronic records of any change in, as an example, vital and emigration status in the Danish population.

Record review

To identify subjects we used the surgical procedures embolectomy in brachial, ulnar, or radial artery, coded according to the Danish Classification of Surgical Procedures (codes: 86823 and 86824) or the Nordic Classification of Surgical Procedures (codes PBE20 and PBE30). We decided to evaluate those two diagnoses jointly, since the embolus can often be found in all three arteries at the same time.

After having identified our subjects, we retrieved their medical records from 30 Danish hospitals, of which 10 have departments of vascular surgery. Records were reviewed by experienced clinicians in cardiovascular medicine (LA, LF). Review data were collected using a detailed standardized form. Embolectomy in brachial, ulnar, or radial artery was considered confirmed when the reviewer verified a description of the surgical procedure in the medical record.

The study was approved by the National Board of Health, the Danish Data Protection Agency, and the heads of the clinical departments.

Analysis

We linked records from the different registries by using the civil registration number. In one case the same operation was registered twice under different civil registration numbers in the Danish National Vascular Registry. One obviously erroneous record among these was therefore deleted from the study. Subjects, whose medical records could not be retrieved for confirmation of the procedure, and records in which embolectomy could not be confirmed were excluded from the analysis of completeness and from the capture–recapture analysis.

The positive predictive value of the embolectomy operation codes in the Danish National Vascular Registry was calculated as a proportion, by dividing the number of registered cases in the Danish National Vascular Registry with a confirmed operation code of embolectomy after medical record review by the total number of retrievable medical records registered in the Danish National Vascular Registry with the operation codes in question.2

To estimate the degree of completeness of the Danish National Vascular Registry, we used as reference the Danish National Registry of Patients after the data were confirmed by review of retrievable medical records. The number of correctly coded cases with embolectomy registered in both the Danish National Vascular Registry and the Danish National Registry of Patients was divided by the total number of identified and correctly coded embolectomies from the Danish National Registry of Patients. The degree of completeness was also calculated separately for the first and second period of registration, namely 1990–1996 and 1997–2002.

To estimate the number of cases that were registered in neither the Danish National Vascular Registry nor in the Danish National Registry of Patients, we used two-source capture–recapture analysis.19 Only data confirmed by retrievable medical record review were used. The maximum likelihood estimate of cases (unobserved cell, Table 1) was calculated as a proportion with the numerator containing the product of the patients registered only in the Danish National Vascular Registry and only in the National Registry of Patients, and as denominator patients registered in both registries.4,19 Confidence intervals (95% CI) around the estimates were calculated according to the binomial distribution. Statistical analyses were performed using STATA 10 (Stata Corporation, College Station, TX, USA).

Table 1

Two-source capture–recapture method

NVR	NRP
	Registered cases	Nonregistered
Registered cases	a = 954	b = 35	a + b
Nonregistered	c = 149	d = (5)	c + d
	a + c	b + d

Notes: Cell numbers present cases with an operation of upper limb embolectomy confirmed by medical record review. The number in brackets presents the estimated number of embolectomy cases not registered in any of the two registries (= bc/a).

Abbreviations: NVR, the Danish National Vascular Registry; NRP, the Danish National Registry of Patients.

Results

We found 1433 incident cases of embolism in both registries jointly during the study period.

There were a total of 1124 cases registered in the Danish National Vascular Registry and we could retrieve 1014 (90.2%) medical records for review (Figure 1). In the National Registry of Patients, there were a total of 1367 cases in the same period, and 1134 (83.1%) medical records could be retrieved for review (Figure 1). After reviewing the medical records from the Danish National Vascular Registry the embolectomy operation was confirmed in 989 records, and in the Danish National Registry of Patients the operation was confirmed in 1103 records. Among patients with confirmed operation there were 954 patients registered in both registers, 149 were registered only in the National Registry of Patients, and 35 were registered only in the Danish National Vascular Registry (Figure 2).

Figure 1

Distribution of medical record retrieval and review for the two registries.

Notes: 1, retrieved and confirmed embolectomy; 2, retrieved but procedure could not be verified; 3, could not be retrieved.

Figure 2

Distribution of registry source for population with confirmed embolectomy.

Notes: 1, Number of patients registered in both the Danish National Vascular Registry and the Danish National Registry of Patients; 2, Number of patients registered only in the Danish National Registry of Patients; 3, Number of patients registered only in the Danish National Vascular Registry; 4, Estimated number of patients not registered.

We examined whether the independent variables, period (≤1996 or > 1996), sex and age, were predictors of missing medical records, and we found that period was a strong predictor, whereas age was a weak predictor, whereas sex was not.

Among patients in whom registration of embolectomy could not be verified, other surgical procedures were identified: operation for arteriovenous fistulas for dialysis (n = 12), embolectomy of the axillar or subclavian artery (n = 8), implantation or thrombectomy of axillobifemoral prosthesis (n = 3), cut-lesion in brachial artery (n = 1), exploration of brachial artery in connection to operation for fracture of the humerus (n = 1), and unspecified exploration of brachial artery (n = 5). For the rest of the cases (n = 13), description of a surgical procedure was missing in the medical record.

The positive predictive value of registration of embolectomy in the Danish National Vascular Registry was 97.5% (95% CI; 96.4–98.4), and the overall degree of completeness of the Danish National Vascular Registry was 86.5% (95% CI; 84.3–88.5).

The estimated number of embolectomy cases not registered in either of those two registries, was five (Figure 2). Thus, the total estimated embolectomy population could be calculated to consist of 1143 individuals (Figure 2).

For the first period of registration, from 1990 to 1996, the degree of completeness was 78.2% (95% CI; 74.4–81.7). For the second period, from 1997 to 2002, the degree of completeness was 93.8% (95% CI; 91.6–95.7).

Discussion

In this study of the degree of completeness and positive predictive value of registration in the Danish National Vascular Registry of the surgical procedures embolectomy in the brachial and in the radial or ulnar artery, we found that the positive predictive value of the procedure coding was high (>90%).

There are no other studies analyzing validity of the procedure coding in the Danish National Vascular Registry, but our results are in agreement with a smaller study analyzing accuracy of surgical codes in the Danish National Vascular Registry, which showed that accuracy of coding of two surgical procedures, an operation for abdominal aortic aneurysm and intrainguinal operation for occlusive disease, was ≥ 90 %.13

Several studies have analyzed validity in other Danish registries. In 2004, a study analyzed the validity of The Danish Hip Arthroplasty Registry and found a high (94%) degree of completeness and a moderate-to-high (84%) positive predictive value.7 Another study of accuracy of coding in the Danish National Registry of Patients found a good agreement between orthopedic surgical codes in the Danish National Registry of Patients and recoding of the procedures (90%); accuracy was higher than for medical diagnoses (75%).12 High positive predictive value and good agreement between coding and recoding of surgical procedures can be explained by precise principles in coding of surgical procedures, which reduce the risk of misclassification.

To assess the number of cases not registered in either of the two registries we used the capture–recapture method.2,19–21 To be able to apply this method it is important to fulfil some assumptions criteria: a) the population is “closed,” b) individuals can be matched between the sources, c) for any single source the probability of ascertainment is the same for each individual, and d) ascertainment of any case by the source is independent.19 All criteria were fulfilled in our study. Owing to a unique 10-digit civil registration number, it was possible to follow and match data for each individual in the Danish population between registries. The Danish Health Care System provides free of charge services for all inhabitants, and it is therefore very likely that individuals with an acute and painful medical condition leading to embolectomy are all admitted to and treated in a public hospital rather than in a private hospital, the records of which are not accessible. And, finally, there is high probability that the two data sources we used are independent, as reporting to the two sources was usually done at different times by different physicians. The estimate of the number of cases not registered in any of the two registries was very low.

The overall degree of completeness of an upper limb embolectomy procedure code in the Danish National Vascular Registry was 86.5%, and the estimated number of cases missing in both registries was low. We found a higher degree of completeness (93.8%) in the second part of the study period. At least two conditions may contribute to this difference: Firstly, the departments of general surgery were more familiar with the Danish National Vascular Registry during the second period. Secondly, during the second period the procedure had become centralized, ie, performed virtually exclusively in large departments of vascular surgery.

There are some limitations to the study. The National Registry of Patients was used as a reference to calculate the degree of completeness despite 35 missing cases in this registry. However, it is to expect that even the best registries are not perfect and that some cases can be missing. We could not retrieve all medical records identified via the Danish National Vascular Registry and the Danish National Registry of Patients (9.8% and 17%, respectively). Most of the missing records were more than 10 years old. The authorities in Denmark require medical record storage for at least 10 years, so older records may have been destroyed owing to storage problems. Otherwise, we could not discover any pattern for the missing records. There is no reason to believe that those missing records are in any way different from the records we could retrieve.

Use of registries for research is growing, and researchers are urged to critically evaluate the quality of the data from the registries prior to epidemiological studies.2,4 Use of already existing data from registries can be a very valuable tool and very cost- and time-effective. Additionally, large sample size and long follow-up period offer the possibility of studying rare diseases and procedures, such as embolism in the upper limb and its treatment, and can provide great precision of estimates. However, existing data from the registries need constant observation of quality and validation.3

In conclusion, the estimated number of embolectomy cases not registered in any of the two registries was low. The overall completeness and positive predictive value of embolectomy in the brachial and in the radial or ulnar artery in the Danish National Vascular Registry was 86.5% and 97.5%, and the registry can be a valuable tool for epidemiological research and quality-monitoring.