Primary hip replacement prostheses and their evidence base: systematic review of literature.

OBJECTIVE: To determine the extent to which prostheses with no readily available evidence to support their use are being implanted in primary total hip arthroplasty.
DESIGN: Systematic review of the literature. DATA SOURCES: The 9th annual report of the National Joint Registry of England and Wales (NJR) was analysed to identify prostheses with an Orthopaedic Data Evaluation Panel rating of "unclassified" or "pre-entry" used in primary total hip arthroplasty in 2011. A systematic review of those prostheses was carried out using PubMed, Cochrane, Embase, OVID, and Google databases. STUDY SELECTION: Prostheses used in primary total hip arthroplasty as published in the NJR's 9th annual report were analysed. Only literature that included the name of the prosthesis was included. Literature yielded in the search results was excluded if it reported animal, non-orthopaedic, non-total hip arthroplasty, or non-device related studies.
RESULTS: The systematic review found that 24% (57/235) of all hip replacement implants available to surgeons in the UK have no evidence for their clinical effectiveness. It also shows that 10,617 (7.8%) of the 136,593 components used in primary hip replacements in 2011 were implanted without readily identifiable evidence of clinical effectiveness. These comprised 157 cemented stems (0.5% of 34,655 implanted), 936 (2.8% of 33,367) uncemented stems, 1732 (7.1% of 24,349) cemented cups, and 7577 (17.1% of 44,222) uncemented cups.
CONCLUSIONS: This study shows that a considerable proportion of prostheses available to orthopaedic surgeons have no readily available evidence of clinical effectiveness to support their use. Concern exists about the current system of device regulation, and the need for a revised process for introducing new orthopaedic devices is highlighted.

Introduction

Medical device regulation has been the subject of recent debate.1 2 3 4 Both professional and public confidence in the system is at a low point. This is particularly true in orthopaedics, where the premature failure of some metal-on-metal hip replacements has added considerably to the global burden of hip revision. As a result, the British Orthopaedic Association has been at the forefront of demanding that a good evidence base accompanies each orthopaedic treatment. In addition, the association’s practice strategy document “Restoring your Mobility” has joined others in calling for radical change in the way medical devices are regulated in the European Union.5 6

In the United States, the regulation of medical devices falls under a single agency, the Food and Drug Agency, in much the same way as drugs are regulated by the European Medicines Agency in the EU. However, medical devices in Europe are evaluated for safety and reliability through any one of 76 notified bodies, which then issue a Conformité Européenne mark, allowing that device to be marketed in Europe.7 All hip replacement prostheses are considered class III devices, which means that the application for approval must include some human clinical investigations,8 although this need not be new research specific to the device if the manufacturer is claiming similarity to an existing product.

Although no clear list of which class III devices have been cleared for use in the United Kingdom or Europe is available, in the case of hip replacement prostheses agencies exist that allow some comparison of available devices. One of these is the National Joint Registry (NJR). This was established in 2001 by the Department of Health and the Welsh Assembly Government to collect information on all hip and knee replacement operations in England and Wales and to monitor the performance of the prostheses used. In April 2011 reporting became mandatory for all National Health Service hospitals, and this now captures data from most of such operations, with compliance rates per hospital varying between 93% and 100% in the NJR’s latest report.9

A second agency is the Orthopaedic Data Evaluation Panel (ODEP), set up as part of the NHS to assess follow-up data for different primary hip replacement prostheses. Manufacturers are requested, but not required, to submit data on their product by using a pro-forma. Prostheses are then classified by the number of years post-implantation that the evidence spans (3, 5, 7, or 10) and by the quality of the data supplied (level A being strong evidence, level B reasonable evidence, and level C weak evidence); 10A is the highest rating available.10 To achieve this 10A benchmark, the prosthesis must have a failure rate of 10% or less at 10 years, a cohort study of more than 500 joints at its start, Kaplan-Meier survivorship data at 10 years, and registry data supporting its use.

Products with less than three years of evidence are classified as “pre-entry” as long as the manufacturers keep the ODEP informed of the post-marketing surveillance data. Inclusion in this pre-entry category does not require evidence from peer reviewed literature at either the pre-clinical or clinical stages of development. Therefore, any data, regardless of whether it has been peer reviewed or not, can be submitted. Prostheses termed “unclassified” have had no evidence submitted by the manufacturers. How many of these “unclassified” and “pre-entry” implants actually have peer reviewed evidence to support their use is unclear, but many are widely available for implantation by any orthopaedic surgeon.

This study aimed to establish the number of primary hip replacement prostheses that have no readily available evidence of clinical effectiveness to support their use and how many are being implanted in clinical practice.

Methods

We analysed the NJR’s 9th annual report (2012) to group primary hip replacement prostheses according to their ODEP rating (fig 1). For prostheses with an ODEP rating of pre-entry or unclassified, we did a systematic review of the literature by using PubMed, Cochrane, Embase, OVID, and Google databases to look for peer reviewed papers of any evidence level relating to the prosthesis in question. We did not search for custom, revision, or discontinued prostheses.

Fig 1 Process used to identify unrated devices and determine evidence levels. NJR=National Joint Registry; ODEP=Orthopaedic Data Evaluation Panel

The search terms and protocol used were “prosthesis name” AND “hip”. The “prosthesis name” was that given in the NJR’s report as the “brand name” and compared with the manufacturer official website. If a discrepancy in the name was found, we used both names individually for the literature search. We did an additional search for each of the prostheses excluding generalised words from the brand name given in the NJR’s report, such as “cementless” or “stem,” to avoid missing potentially relevant articles.

Two researchers (FK-P and TTM) did the literature research and independently reviewed all results to establish the highest level of evidence for each of the prostheses; a third researcher (AMA) resolved any discrepancies. Titles and abstracts were reviewed, and those that were potentially relevant to the device in question were included. We defined evidence as peer reviewed publications in which the clinical effectiveness of a particular device was assessed. We excluded animal, non-orthopaedic, non-primary hip arthroplasty, and non-device specific studies (fig 1).

We then gave the selected papers an evidence level rating according to the simplified evidence level table from the Centre for Evidence-Based Medicine, Oxford (box 1),11 and we established the highest level evidence available for each device. If no suitable evidence could be identified, we contacted the manufacturers and asked them to provide some data on their prosthesis. Those that responded with papers were rated; if some data were provided (for example, details of tensile strength or principles to support the device’s use), this would earn a level 5 evidence rating (expert opinion without explicit critical appraisal/pre-clinical biomechanical data).

1a: Systematic reviews of randomised controlled trials

1b: Individual randomised controlled trials

1c: All or none randomised controlled trials

2a: Systematic reviews of cohort studies

2b: Individual cohort study or low quality randomised controlled trials

2c: Outcomes’ research; ecological studies

3a: Systematic review of case-control studies

3b: Individual case-control study

4: Case series

5: Expert opinion without explicit critical appraisal/pre-clinical biomechanical data

We then analysed the collected evidence to determine the number of brands implanted with no evidence of clinical effectiveness and, additionally, the number implanted with no evidence at any level. Those implants with only level 5 data were excluded from the analysis, as we believed this level to be inadequate for clinical decision making. We then cross referenced this information with the published NJR9 prostheses tables to find the numbers of prostheses actually implanted into patients in each of these two categories.

Statistical methods

Statistical analysis in this study focused on descriptive statistics. We tabulated frequencies for the appropriate implants. We calculated percentages by using the total number of medical devices in the relevant population as the denominator. We report the findings according to the PRISMA guidelines.12

Results

Data from 9th annual NJR report (2012)

The NJR’s report shows that 142 different brands of femoral stems (57 cemented, 85 uncemented) and 119 different brands of acetabular cups (48 cemented, 71 cementless) were used in primary total hip replacement procedures in 2011 (table 1). The proportion of components implanted that achieved the optimal ODEP rating of 10A varied between the different types of prosthesis; 85% of cemented stems implanted achieved a 10A rating, dropping to 72% of cementless stems, 40% of cemented cups, and only 3% of cementless cups (table 2). Forty eight per cent (126/261) of prosthesis brands implanted in primary hip replacements in 2011 were categorised as having an ODEP rating of pre-entry or unclassified. On closer inspection of the published NJR prostheses tables, we determined eight of these brands to be revision implants, leaving 118 unrated primary prosthesis brands included for further analysis (table 1).

Table 1

 Breakdown of Orthopaedic Data Evaluation Panel’s rating for each device category from 9th National Joint Registry of England and Wales report. Values are numbers (percentages)

	Cemented stem (n=57)	Uncemented stem (n=85)	Cemented cup (n=48)	Uncemented cup (n=71)	Total (n=261)
ODEP rated
10A	14	16	11	9	50
10B	2	1	1	2	6
10C	2	2	0	1	5
7A	3	3	1	4	11
7B	2	1	0	1	4
5A	5	4	2	12	23
5B	0	1	2	0	3
3A	4	7	1	2	14
3B	0	0	0	1	1
Total	32	35	18	32	117 (45)
Unrated
Unclassified	14	33	23	24	94
Pre-entry	3	13	3	13	32
Custom/revision/discontinued	8	4	4	2	18
Total	25	50	30	39	144 (55)

Table 2

 Numbers (percentages) of prostheses implanted, 2011

Rating	Cemented stem	Uncemented stem	Cemented cup	Uncemented cup	Total
10A	30 689 (88.6)	23 920 (71.7)	9751 (40.0)	1531 (3.5)	65 891
10B	520 (1.5)	147 (0.4)	41 (0.2)	35 (0.1)	743 (0.6)
10C	118 (0.3)	142 (0.4)	0 (0)	69 (0.2)	329 (0.2)
7A	260 (0.8)	222 (0.7)	223 (0.9)	18 639 (42.1)	19 344 (14.2)
7B	164 (0.5)	3514 (10.5)	0 (0)	6 (<0.1)	3684 (2.7)
5A	2172 (6.3)	1870 (5.6)	8436 (34.6)	10 530 (23.8)	23 008 (16.9)
5B	0 (0)	49 (0.1)	1171 (4.8)	0 (0)	1220 (0.9)
3A	419 (1.2)	639 (1.9)	1735 (7.1)	3405 (7.7)	6198 (4.5)
3B	0 (0)	0 (0)	0 (0)	7 (<0.1)	7 (<0.1)
Unclassified	204 (0.6)	1061 (3.2)	2746 (11.3)	1434 (3.2)	5445 (4.0)
Pre-entry	84 (0.2)	1775 (5.3)	206 (0.8)	8552 (19.3)	10 617 (7.8)
Custom/revision/discontinued	25 (0.1)	28 (0.1)	40 (0.2)	14 (<0.1)	107 (0.1)
Total	34 655 (100)	33 367 (100)	24 349 (100)	44 222 (100)	136 593 (100)

Available evidence for pre-entry and unclassified components

A literature search was conducted for these 118 brands of prosthesis. In total, 8111 papers were reviewed by title and abstract. After application of exclusion criteria, 368 papers were reviewed in full and a further 211 were then discarded, leaving 157 relevant papers. These were classified according to the level of evidence they contained (fig 2). Four of 157 papers had a discrepancy in the level of evidence between the two initial researchers, which were resolved by a third researcher. The inter-observer correlation between the researchers who did the systematic review was 97%. Unpublished data subsequently provided by manufacturers changed the evidence base rating on one cemented stem, two uncemented stem, and three uncemented cup brands.

Fig 2 Flow chart of literature search and evidence level classification

We could identify no evidence of clinical effectiveness for 48% (57/118) of pre-entry and unclassified prosthesis brands. Excluding custom, revision, and discontinued implants, these accounted for 24% (57/235) of the total number of primary hip replacement prostheses brands listed in the NJR’s report (box 2; fig 3). If we counted level 5 evidence (for example, biomechanical data showing equivalence to pre-existing devices), this still left 42% (49/118) of the pre-entry and unclassified prosthesis brands and 21% (49/235) of all brands used bereft of any identifiable evidence to support their use.

Fig 3 Percentage of available prosthesis brands with no evidence of clinical effectiveness in 2011 (top) and percentage of prostheses implanted with no evidence of clinical effectiveness in 2011 (bottom)

Box 2 List of prostheses implanted in 2011 with no readily available evidence of clinical effectiveness

Unclassified devices

Cemented stem

Excia Cemented (B Braun/Aesculap)

Kinectiv Cemented (Zimmer)

Edinburgh* (Implants International)

Trilliance† (B Braun/Aesculap)

Answer (Biomet)

CTI Cemented Stem (Corin); no longer available

Response (DePuy)

Wroblewski Resection (DePuy)

Kinectiv Cemented (Zimmer)

Uncemented stem

H-Max (Lima)

FH Modular (FH Orthopaedics)

MiniMax (Medacta UK Ltd)

Harmony (Symbios SA)

Lima SL (Lima)

LPS (DePuy)

Euros Cementless (Euros)

Cemented cup

Exeter RimFit (Stryker)

CMK Cemented Cup‡ (Biomet)

Exceed ABT Cemented (Biomet)

CCB (Implants International)

Edinburgh Cup* (Implants International)

Alfa Cemented Cup (B Braun/Aesculap); discontinued 2008

PE (Symbios SA)

Luna (Amplitude)

Charnley KS (DePuy)

Cone Cup (Medacta UK Ltd)

Snap Fit (Waldemar Link)

Uncemented cup

April (Symbios SA)

Gyros (DePuy)

Regenerex Ringloc + (Biomet)

Novation (Exatech)

Beta Cup (Waldemar Link)

Split Cup (Surgicraft)

Restoration ADM Cup (Stryker)

MMC Cup (Zimmer)

Maxera (Zimmer)

Dynasty (Wright Medical UK Ltd)

Sirius Cementless Cup (Euros)

Pre-entry devices

Cemented stem

Corail Cemented (DePuy)

Uncemented stem

Metafix Stem* (Corin)

MiniHip* (Corin)

Silent (DePuy)

Amoda (Comis Orthopaedics)

Trabecular Cementless Stem (Zimmer)

Novation Element Stem (Exactech)

Novation Stem (Exactech)

Cemented cup

Apollo‡ (Biomet)

Polarcup Cemented (Endo Plus (UK) Ltd)

Uncemented cup

CSF plus* (Joint Replacement Instrumentation Ltd)

Continuum (Zimmer)

Trinity* (Corin)

Procotyl (Wright Medical)

DeltaMotion* (DePuy)

Versafit* (Medacta UK Ltd)

seleXys PC (Mathys Orthopaedics Ltd)

seleXys TH (Mathys Orthopaedics Ltd)

DeltaMotion (Lima)

seleXys TPS (Mathys Orthopaedics Ltd)

*Awarded 3A status in NJR10 (2013)

†Manufacturer states under post-marketing surveillance

‡Awarded 5A status in NJR10 (2013)

Use of component with no available evidence

Applying the results of our systematic review to the NJR’s 9th report data showed that 10 617 prostheses without available evidence of clinical effectiveness to support their use were implanted into patients in 2011 (7.8% of the 136 593 prostheses included in the report) (table 2). This number comprised 157 cemented stems, 936 uncemented stems, 1732 cemented cups, and 7577 uncemented cups.

Cemented stems represented the group with the lowest proportion of devices implanted without available evidence of clinical effectiveness (0.5%; 157/34 655), followed by uncemented stems (2.8%; 936/33 367). Higher numbers of cups were implanted without such evidence; 7.1% (1732/24 349) of cemented cups, and 17.1% (7577/44 222) of cementless cups had no available evidence of clinical effectiveness to support their use (table 3; fig 3).

Table 3

 Summary of highest level of evidence found for unrated (unclassified and pre-entry) brands by prosthesis type. Values in parentheses are numbers implanted

Evidence level	Cemented stem	Uncemented stem	Cemented cup	Uncemented cup	Total
1A	—	—	—	—
1B	1 (20)	2 (218)	—	3 (47)	6 (285)
1C	—	—	—	—	—
2A	—	—	—	—	—
2B	—	3 (552)	—	2 (1,213)	5 (1,765)
2C	—	—	1 (1)	—	1 (1)
3A	—	—	—	—	—
3B	1 (36)	1 (9)	1 (149)	—	3 (194)
4	6 (190)	18 (956)	11 (1069)	7 (1726)	42 (3941)
No of prostheses available with no evidence	9 (157); 9 unclassified	14 (936); 7 pre-entry; 7 unclassified	13 (1732); 2 pre-entry; 11 unclassified	21 (7577); 10 pre-entry; 11 unclassified	57 (10 617); 20 pre-entry; 37 unclassified
No of prostheses implanted with no evidence	0.5% of those implanted (157 of 34 655)	2.8% of those implanted (936 of 33 367)	7.1% of those implanted (1732 of 24 349)	17.1% of those implanted (7577 of 44 222)	7.8% of those implanted (10 617 of 136 593)

Revision devices have been excluded (2 cemented stems, 3 uncemented stems, and 3 uncemented cups).

Discussion

This systematic review of the literature shows that 8% of all primary hip replacement prostheses implanted in 2011 and recorded by the National Joint Registry (NJR) had no readily available evidence relating to their safety or effectiveness. This is likely to be an underestimation of the true problem, as much of the evidence that does exist for the other unrated prostheses is of low quality or relates to short term outcomes only. This is of great concern, particularly in light of the widespread publicity surrounding recent safety problems with regard to some resurfacing and other large diameter metal-on-metal joint replacements.13

Evidence ratings

The ODEP system of grading primary hip components offers clinicians a simplified, device specific rating for clear comparison of devices’ performance on relevant clinical criteria. It is troubling to note that 45% of the brands available for primary hip replacement have no ODEP rating, meaning, according to the ODEP guidelines, that they should be implanted only as part of a trial. What is more heartening is that despite the large number of unrated brands available, the most widely used prostheses are highly rated by the ODEP system, implying that most orthopaedic surgeons use prostheses that are supported by a good evidence base.

Why some surgeons would choose to use prostheses with no available evidence to support their use outside of a research study is not clear. Possible reasons include the introduction of new prostheses by manufacturers who have well established and trusted implants available. These new products are then judged, perhaps erroneously, to be of the same high standard in terms of longevity and clinical outcome. Another situation may arise whereby a well established prosthesis is “improved” by way of a minor modification. Unfortunately, numerous examples exist of minor design changes having disastrous effects on implants’ longevity.14 15

Murray, Carr, and Bulstrode reviewed the evidence for total hip replacement in 1995.16 They found that only 30% of hip replacements available had any evidence supporting their use.16 At the time, they advocated the need for greatly improved collection of evidence and proposed improvements in the regulation of implants and devices. Despite the recommendations of this and other groups,4 17 the evidence base in relation to hip replacements has improved only marginally over the past 18 years. The reasons for this are unclear. It may be related to the rapid expansion in the number of devices introduced onto the market during the past two decades as demand for hip arthroplasty increases worldwide. In 1996 there were 62 primary hip replacement components on the market in the UK; in 2011 there were 261.[9 ] 16 In addition, the quality of studies in the orthopaedic literature is of a generally low standard. Our study has identified only six randomised control trials in the literature, compared with the 265 implants currently available. Most of the studies identified were case series. Properly conducted randomised controlled trials are more difficult to run with orthopaedic implants than with drugs or other interventions, and this may also have contributed to the paucity of good quality evidence in this sphere.

Although the ODEP already provides implant ratings based on levels of evidence, manufacturers are not required to submit published negative evidence in the case of either pre-entry or unclassified prostheses. Therefore, a device could sit in either category despite several negative peer reviewed publications. This systematic review has shown that 64% of unclassified prostheses have published peer reviewed evidence that may not have been submitted to the ODEP.

Regulatory process

The regulation process also seems to be entirely inadequate. The award of a Conformité Européenne mark is conditional on a device meeting a series of laboratory based standards that may not equate to the safety or effectiveness of an implant in patients. A Conformité Européenne mark may also be awarded in cases of “existing similarity,” where the new device closely resembles an existing design. However, as discussed above, small changes in design have been shown to have major deleterious effects on an implant’s clinical effectiveness or lifespan.14 15

The NJR is effective in auditing current practice but was designed to monitor the success of implants in relation to when and how often they need to be replaced or revised. Thus, although patient reported outcome measures are due to be introduced as part of the annual report, in its current form the NJR’s report does not recognise problems with implants until they are revised and so may not be the best tool for evaluating new prostheses. Most implants are typically introduced in small numbers initially, which makes outliers difficult or impossible to detect.18 This is shown by the recent problems with some metal-on-metal joint replacements, which were identified in single centre cohort studies more than three years before the NJR identified them as outliers,19 20 Arguably, well designed controlled studies would have identified the problem even more quickly with fewer patients experiencing the adverse consequences of a substandard design. In addition, the NJR lists only implants that are in open use and for which reports are submitted. Consequently, devices that are available to some surgeons but have not yet been made available on the open market do not appear in the NJR’s report. These factors can lead to a delay in the identification of failing implants. The National Institute for Health and Care Excellence (NICE) suggests that the use of more refined outcome measures such as radiostereometric analysis, which aims to identify early loosening of implants by using bi-planar radiographs, may help to detect early problems.

The phased introduction of devices, combined with the use of surrogate outcome measures, has been called for to provide early identification of poorly performing implants.18 This correlates with the IDEAL framework ensuring that the introduction of new devices is controlled and regulated in phases. The future of medical device regulation needs to be a careful balance between the requirement to facilitate innovation and the imperative to safeguard patients. Following the example of pharmaceutical regulation by implementing a phased introduction of new orthopaedic implants would seem prudent. It has been proposed that innovative technologies should be made available in a few specialist units, where the evidence can be objectively gathered and any problems with implants’ resilience identified quickly using surrogate outcome measures.17

The counter argument debates the damage over-regulation can do to innovation and development in the field of medicine.21 The cost of medical implants and devices may also rise if they are required to undergo lengthy pre-clinical and clinical testing. However, significant cost savings may accrue if the number of devices on the market were limited to only those devices with a solid evidence base for their use. Tackling this question requires a delicate balance.

Many of the concerns in this process relate to the evidence required to market an implant or device. Public availability of a list of current medical implants and devices, including pre-marketing data and peer reviewed publications, is needed. This may improve transparency in the early stages of implant introduction, allowing surgeons and patients to make more informed decisions.

Limitations of study

A major limitation to our study relates to the requirement that a prosthesis be specifically named in a publication to meet our inclusion criteria. Relevant published evidence may therefore not have been identified, for instance, if a specific implant was simply referred to as “an uncemented acetabular cup” or as a “proximal loading femoral stem.” However, we felt that the explicit naming of a prosthesis is necessary if surgeons or commissioners are to locate evidence that can inform their decision making as to the use of a particular prosthesis. Nevertheless, evidence for some implants may have been missed if the device had undergone a change of brand name since evidence was published. However, in all cases in which no evidence could be found, we contacted the manufacturers directly to request supportive data for the use of their implant, which gave the opportunity for missed evidence to be brought to our attention.

Another limitation is that the evidence for some early phase implants will not have been detected if they are part of ongoing prospective cohort studies or randomised controlled trials that have yet to report. We have included, where identified, “pre-entry” devices in our analysis, and these prostheses are likely to be undergoing prospective studies and so would not yet have published evidence available in the literature. However, the number of prostheses implanted as part of prospective and unreported clinical studies is likely to represent only a small proportion of the 10 617 devices implanted with no available evidence.

The evidence presented in this study relates to prostheses implanted in the UK. However, we believe the results of this study can be applied to other healthcare settings, given that most of the prostheses in our systematic review are available in most other developed countries, in Europe, Australasia, and the United States.

Conclusion

NICE has set a clear benchmark of an ODEP rating of 10C as the minimum clinical standard it recommends for general implantation.22 Our review has determined that only 49% (118/261) of prostheses implanted overall achieved a 10A/B/C rating and that almost one in four prosthesis brands available to surgeons have no evidence to support their use. Although these brands are used relatively infrequently, at least 7% of all prostheses implanted lack evidence of clinical effectiveness or longevity. This study shows that the need still exists for an improved and more rigorous approach to regulation of devices to avoid devices with no available evidence being used in a widespread and uncontrolled manner.

The high failure rate of some metal-on-metal hip replacements has highlighted the need for an adequate evidence base for orthopaedic implants

Many implants are available to orthopaedic surgeons, but how many of these have evidence of clinical effectiveness to support their use is not known

A quarter of prostheses available to the surgeon for use in primary total hip arthroplasty in the UK have no available evidence of clinical effectiveness to support their use

Almost 8% of all primary hip replacement prostheses implanted in 2011 had no readily available evidence relating to their safety or effectiveness