Are all metal-on-metal hip revision operations contributing to the National Joint Registry implant survival curves? : a study comparing the London Implant Retrieval Centre and National Joint Registry datasets.

AIMS: The National Joint Registry for England, Wales and Northern Ireland (NJR) has extended its scope to report on hospital, surgeon and implant performance. Data linkage of the NJR to the London Implant Retrieval Centre (LIRC) has previously evaluated data quality for hip primary procedures, but did not assess revision records.
METHODS: We analysed metal-on-metal hip revision procedures performed between 2003 and 2013. A total of 69 929 revision procedures from the NJR and 929 revised pairs of components from the LIRC were included.
RESULTS: We were able to link 716 (77.1%) revision procedures on the NJR to the LIRC. This meant that 213 (22.9%) revision procedures at the LIRC could not be identified on the NJR. We found that 349 (37.6%) explants at the LIRC completed the full linkage process to both NJR primary and revision databases. Data completion was excellent (> 99.9%) for revision procedures reported to the NJR. DISCUSSION: This study has shown that only approximately one third of retrieved components at the LIRC, contributed to survival curves on the NJR. We recommend prospective registry-retrieval linkage as a tool to feedback missing and erroneous data to the NJR and improve data quality. TAKE HOME MESSAGE: Prospective Registry - retrieval linkage is a simple tool to evaluate and improve data quality on the NJR. ©2016 Sabah et al.

There has been an information revolution in orthopaedics[1] and we are beginning to realise the opportunities that analysis of large data sets can offer in planning the provision of health care and to study disease patterns. However, failings may also be present in these large datasets.[2]

Registry data can be relied upon to engender debate,[3] not least with the recent publication of surgeon- and unit-level data.[4] As we rely on registry data more frequently to inform practice, there has rightly been a renewed focus on data quality. The National Joint Registry of England, Wales and Northern Ireland (NJR) has made this a key objective in its Strategic Plan for 2013 to 2016.[5] The NJR has helped facilitate the recent validation of primary metal-on-metal hip arthroplasty procedures by the London Implant Retrieval Centre (LIRC).[6]

The objectives of this current study were to continue the examination of data quality of the NJR and to demonstrate the value of registry: retrieval centre linkage and to validate data quality on metal-on-metal hip revision procedures.

Patients and Methods

We performed a cross-sectional survey of the NJR and LIRC databases on 5 November 2013. The study received ethical approval from our institutional review and ethics board.

Data linkage between LIRC and NJR databases

This is illustrated in Figure 1. We performed linkage to the NJR de-anonymisation table, NJR primary table and NJR revision table. These contain data on patient identifiable information (for those patients who gave their permission on an NJR Patient Consent form), primary procedures and revision procedures, respectively.

Flowchart showing potential linkage of component data stored on the London Implant Retrieval Centre (LIRC), to patient and procedure data stored on the National Joint Registry (NJR) revision database.

A total of 69 929 hip revision procedures associated with a ten-digit NHS number[7] and patient consent were identified from the NJR database. More than 3000 orthopaedic implants have been collected by the LIRC since it was established in 2006, encompassing hip, knee and ankle replacements and spinal components. A total of 929 explanted pairs of hip components were included in the study. Hip revision procedures performed within the geographical and temporal remit for the NJR were eligible for inclusion, independent of the details of the primary procedure. Records missing key data for linkage were excluded.

The linkage process involved two stages: firstly, identifying the LIRC patient on the NJR database (patient-level linkage) and secondly, identifying the correct revision procedure for that patient (procedure-level linkage).

Patient-level linkage of NJR and LIRC records was dependent on the NHS number. Verification of patient’s name and date of birth was performed using the NHS Spine[8] which is a directory of demographic data for 80 million patients in the United Kingdom.[6] Procedure-level linkage was performed for patients using an exact match for the date of revision surgery and implant side. This process was repeated to increase the number of linked procedures, by relaxing date constraints on the revision operation to +/- 14 days.

Statistical analysis and validation of data

Detailed data validation was performed for a subset of the fields in the NJR revision database. The paired fields for the NJR revision database and LIRC are shown in Table I. No imputation was performed for missing data. Linked records were validated using the LIRC as the reference standard, with ‘error’ defined as ‘a discrepancy between the NJR revision database and LIRC field entry’. Statistical analyses were performed using SPSS Statistics version 22.0 (IBM, Armonk, New York) using our previously published methods.[6]

Percentage rates of error were calculated by dividing the absolute number of errors by the total number of data points. Binomial distribution was used to calculate 95% confidence intervals (CI), and Fisher’s exact test applied to 2 × 2 contingency tables where necessary.[9,10] All statistical tests were two-tailed and significance was assumed at a p-value of < 0.05.

Results

This is shown in Figure 1. All 929 LIRC patients were found on the NJR de-anonymisation database. This indicated that all patients had consented to at least one joint replacement procedure being recorded on the NJR. However, this may have been for a different component to that held at the LIRC (e.g., a different joint, the opposite side, or an earlier/later procedure). Patient-level linkage identified 857 patients (92.2%) on the NJR revision database.

Procedure-level linkage that required an exact match on date of revision and implant side linked 654 procedures (70.4%). This was increased to 716 procedures (77.1%) when date constraints were relaxed to +/- 14 days. All revision procedures on the LIRC database were performed in England and Wales after the NJR commenced on 1 April 2003, or in Northern Ireland after 1 February 2013. Therefore, none were excluded. A total of 213 procedures (22.9%) from 73 surgical units that were expected to be recorded in the NJR revision database were not linked.

How many component pairs contributed to survival curves?

This is shown in Figure 2. In all, 349 out of 929 procedures (37.6%) completed the full linkage process from the LIRC to both NJR primary and revision databases. This meant that 367 patients out of the 716 linked revisions were not linked to a primary procedure on the NJR. It was identified that only 43 of these patients had their primary arthroplasty performed before the inception of the NJR.

An outline of how consent and the level of handling of data influence the journey of data through the National Joint Registry. HDM, hospital data manager; NJR, National Joint Registry for England, Wales and Northern Ireland; DB, database; H1 form, NJR data collection form for primary hip arthroplasties; H2 form, NJR data collection form for single-stage revision, hip revisions and hip excision arthroplasty.

Our earlier paper revealed a figure of 42.7% (397 out of 929 procedures) could be derived for the full linkage process.[6] This represented 476 LIRC components linked to a primary procedure minus 79 procedures where the NJR outcome coding was incorrect. The discrepancy represents subtle differences in the linkage methodologies between the two studies.

Detailed data validation

This is provided for the NJR revision database in Table II. There were no duplicated records. The NJR had near perfect rates of completion (> 99.99%) for six of its data fields. Lower completion rates were observed for details relating to primary surgery: procedure date or year was present in 46 110 out of 69 929 (66.0%) hip revision procedures and surgical unit in 40 336 out of 69 929 (57.7%) hip revision procedures. Revision component fields had variable completion rates. This may reflect component usage, rather than missing data. For example, an exchange of acetabular liner would be correct to leave shell, head, femoral component and cement fields blank.

Validation of linked records

This is presented in Table III. Data related to the revision procedure had nearly perfect completion and accuracy. Quality of data for primary procedures was poorer. The primary procedure date (accurate to within 12 months) was available in 238 out of 716 (33.2%) records, and there was a discrepancy in 22 out of 238 (9.24%). The primary procedure hospital was present in 397 out of 716 (55.4%) records and discrepant in 56 out of 397 (14.1%) instances. None of the discrepant primary hospitals could be attributed to NHS Trust mergers or other administrative changes. The small error rate for NJR Outcome Code of three out of 352 (0.85%) procedures demonstrated that the automatic system the NJR uses to link procedures is reproducible.

Discussion

This study found that only 37.6% of retrieved component pairs at the LIRC contributed to survival curves on the NJR. We have shown that 213 revision procedures (22.9%) and 364 primary procedures (39.2%) could not be identified on the NJR. This suggests that current NJR data on failure rates may be vulnerable to missing data. The most likely explanation for this appears to be the poor rate of consent, compliance and linkability during the early years of the NJR. We recommend that the NJR provide outcome data only for the periods where it has achieved excellent data collection. We would also advocate for registry: retrieval linkage to become an integral component of the NJR Data Quality Strategy. This would enable feedback on errors and missing data.

This study is the continuation of our independent assessment of data quality for metal-on-metal hip procedures recorded on the NJR. We have turned our attention to revision procedures, having previously focussed on primaries.[6] This study and analysis highlights the importance of matching retrievals and information directly from revision surgeons, and not solely to rely on registry data sets.

We have confirmed our previous finding that NJR data capture is improving.[6] The LIRC was founded in late 2006, therefore, revisions performed before this date and recorded on the NJR, have not been recruited into this study. Figure 3 shows the marked improvement in data linkage over time.

Chart showing that 716 out of 929 eligible procedures were linked to a revision procedure recorded with the National Joint Registry. Rates of linkage were relatively poor before 2008, and have remained consistent since 2010.

We found that for the procedures the NJR did record, data quality was excellent. There was no missing data for surgical unit, date of revision, procedure type and implant side. This reflects the engagement from surgeons and Hospital Data Managers (HDMs) and the high quality of the NJR database infrastructure. Missing and discrepant data related to the primary procedure on the NJR revision database should be interpreted in the correct context. First, for linked procedures, these data are available with exceptional quality on the NJR primary database. Second, primary data gathered at the time of revision are not used in NJR survival analyses.

Our study should help to de-mystify how procedures contribute to the survival curves on the NJR. The NJR defines “revision” as “an operation to remove or replace one or more components of a joint prosthesis”.[11] It follows that a simple washout or joint reduction following dislocation, does not meet this definition, whereas replacement of acetabular liner does.

The NJR primary hip arthroplasty (H1) or revision hip arthroplasty (H2) data collection forms are usually completed by the surgeon and either self-entered onto an electronic system by the surgeon or at a later date by the HDM. Patient information is then checked centrally using the Personal Demographics Service,[12] which is part of NHS Spine.[8] Patients whose details do not match are re-checked with the HDM, but may need to be excluded. This process can also be used to identify the NHS number in cases where it has not been provided.

The NJR links primary and revision procedures automatically. Patients are linked using the NHS number. Procedures are linked if the joint (e.g. hip, knee, shoulder, etc.) and side are the same and the revision is performed after the primary surgical date. The series of steps that need to fall into place for primary and revision operations to be linked are shown in Figure 3. This highlights some important practical considerations for surgeons. The most important is that, regardless of patient consent status, surgeons should be completing data collection forms for all joint arthroplasty procedures with which they are involved.

In the scenario where a patient withholds consent, the NJR records procedure and component information in association with a patient hospital identifier, but no patient identifiable information. These records are not eligible for any form of linkage either within the NJR or to an external dataset.

Where patient consent is positive or unknown, data can be used for survival analysis. In the latter case, Section 251 of the NHS Act 2006[13] allows the NJR to collect patient details. For 2013/2014 the rate of patient consent was 91.8%.[14]

The other practical consideration for the NJR is the content of the H2 form. The NJR needs to be provided with sufficient information to identify the relevant primary procedure for the patient. We have shown in our previous report[6] that component batch and category numbers can be used in this linkage process. However, it is clear that being able to distinguish these from an explanted prostheis or manufacturers’ component identification labels is not always straightforward. We would advocate for this to be delegated to retrieval centres. In addition, implant companies could do more to educate surgeons about these identifiers. We would support an amendment to the H2 form so that the revising surgeon could inform the NJR of the component reference number for prosthesis they have removed.

We recognise the limitations of this paper. We have only analysed a sample of metal-on-metal hip procedures and have no data to support whether our results can be generalised to other types of joint replacement recorded on the NJR.

We do not believe that our results have been significantly skewed by a few large centres not contributing data to the NJR. We found that the 213 missing revision procedures were shared among 73 surgical units.

We acknowledge that we do not have full source verification for LIRC records. We have used the term ‘discrepancy’ rather than ‘error’ to reflect fields where we are not completely certain of the origin of the disagreement. However, surgeons who contribute prostheses to the LIRC will be familiar with the rigor of our data collection methods. We routinely request objective evidence (e.g. operation notes, imaging, microbiology test results) to support our data collection forms.

We recognise that retrieval centres receive only a proportion of revision cases. The NJR has highlighted Local Data Audit as a pillar of its Data Quality Strategy. We endorse this approach and are working on methodology that could be applied in other centres across the country.

The extent of information collated in a registry has been defined as the Level of the data (Table IV).[15] The NJR currently records Level III data (i.e. data on peri-operative care and complications, and patient-reported outcome measures in addition to basic patient, surgeon and implant identifiers) The Beyond Compliance initiative[16] has started to generate Level IV data. Prospective Registry:Retrieval Linkage is an opportunity to provide Level V data.[15] The task ahead for the NJR is to turn the largest registry in the world, into the best registry in the world.

Table I

Fields for validation

Field name	Variable type	Comment
Primary procedure date	Continuous	-
Primary procedure hospital	Categorical	-
Implant side	Dichotomous	Left or right
Revision procedure date	Continuous	-
Revision procedure hospital	Categorical	-
NJR outcome code	Categorical	Revised, unrevised or dead

Description of field level linkage between National Joint Registry (NJR) and London Implant Retrieval Centre datasets

Table II

Detailed validation of National Joint Registry (NJR) revision database

Field	Valid	OOR	Missing	Missing rate (%)	Mean/count	SD	Max	Min
Patient age at surgery	69 928	0	1	0.00	69.67	11.938	108	14
Date of birth	69 929	0	0	0.00	-	-	-	-
Primary procedure date	46 110	19	23 800	34.03	-	-	-	-
Primary procedure hospital	40 336	0	29 593	57.68	478 hospitals	-	-	-
Implant side	69 929	0	0	0.00	Left: 33162 Right: 36767	-	-	-
Revision procedure date	69 928	1	0	0.00	-	-	-	-
Revision procedure hospital	69 929	0	0	0.00	431 units	-	-	-
Acetabular component manufacturer	51 143	0	18 786	26.86	36 manufacturers	-	-	-
Acetabular component brand	51 015	0	18 914	27.05	164 brands	-	-	-
Liner component manufacturer	40 400	0	29 529	42.23	29 manufacturers	-	-	-
Liner component brand	1 047	0	68 882	98.50	-	-	-	-
Head component manufacturer	57 162	0	12 767	18.26	34 manufacturers	-	-	-
Head component brand	746	0	69 183	98.93	8 brands	-	-	-
Stem manufacturer	29 523	0	40 406	57.78	29 manufacturers	-	-	-
Stem brand	29 458	0	40 471	57.87	166 brands	-	-	-
Cement manufacturer	28 706	0	41 223	58.95	9 manufacturers	-	-	-
Cement brand	28 706	0	41 223	58.95	27 brands	-	-	-
Revision procedure type	69 929	0	0	0.00	Excision arthroplasty: 535 Single stage revision: 58 972 Hip stage 1 of 2: 3 991 Hip stage 2 of 2: 4 862 Hip revision (Stage 2 of 2): 1 569	-	-	-
Patient position	65 405	0	4 524	6.47	Lateral: 61 037 Supine: 4 368	-	-	-

Descriptive statistics for 69 929 records of metal-on-metal revision procedures recorded on the National Joint Registry from 1 April 2003 to 31 December 2013. Records without NHS numbers were excluded, as they were not eligible for linkage Missing rate %, number missing/total number of records × 100; Max, maximum; Min, minimum; OOR, out of range; sd, standard deviation; Valid, total number of records (n = 69 929); number OOR, number missing; manufacturer, company that makes goods for sale (e.g. Smith & Nephew); brand, a type of product manufactured by a particular company under a particular name (e.g. BHR Resurfacing Head)

Table III

Validation of linked records

Field	Valid	Missing	Missing rate (%)	Discrepancies	Rate of discrepancy (%)	Concordant
Primary procedure date or year	238	478	66.76	22	9.24	216
Primary procedure hospital	397	319	44.55	56	14.11	341
Implant side*	716	0	0.00	0	0.00	716
Revision procedure date*	716	0	0.00	0	0.00	716
Revision procedure hospital	715	1	0.14	0	0.00	715
NJR outcome code	352	364	50.84	3	0.85	349

* Fields used in the linkage algorithm Validation of 716 linked revision procedures. London Implant Retrieval Centre data were used as the reference. Data on primary procedures recorded on the National Joint Registry (NJR) revision database had poor completion and accuracy

Table IV

Levels of registry data[15]

Level of data	Data	Purpose
I	Patient, surgeon and hospital identifiers	Monitoring revision rates
	Procedure data
II	Patient factors	Assessment of complications
	Comorbidities
	Surgical data
	Peri-operative care
	Complications
III	Patient-reported outcome measures	Capturing the patient perspective
		Identification of risk factors for poor outcome
		Assessment of health improvement
		Cost-effective analysis
IV	Radiographs	Detection of subclinical implant failure
V	Explant analysis	Forensic examination of explants to determine cause of failure

This table details the five levels of Registry data. The National Joint Registry currently records Level II data. This could be improved to Level V data, through registry-retrieval linkage