Adverse Event Recording and Reporting in Clinical Trials Comparing Lumbar Disk Replacement with Lumbar Fusion: A Systematic Review.

Study Design Systematic review. Objectives (1) To compare the quality of adverse event (AE) methodology and reporting among randomized trials comparing lumbar fusion with lumbar total disk replacement (TDR) using established AE reporting systems; (2) to compare the AEs and reoperations of lumbar spinal fusion with those from lumbar TDR; (3) to make recommendations on how to report AEs in randomized controlled trials (RCTs) so that surgeons and patients have more-detailed and comprehensive information when making treatment decisions. Methods A systematic search of PubMed, the Cochrane collaboration database, and the National Guideline Clearinghouse through May 2015 was conducted. Randomized controlled trials with at least 2 years of follow-up comparing lumbar artificial disk replacement with lumbar fusion were included. Patients were required to have axial or mechanical low back pain of ≥3 months' duration due to degenerative joint disease defined as degenerative disk disease, facet joint disease, or spondylosis. Outcomes included the quality of AE acquisition methodology and results reporting, and AEs were defined as those secondary to the procedure and reoperations. Individual and pooled relative risks and their 95% confidence intervals comparing lumbar TDR with fusion were calculated. Results RCTs demonstrated a generally poor description of methods for assessing AEs. There was a consistent lack of clear definition or grading for these events. Furthermore, there was a high degree of variation in reporting of surgery-related AEs. Most studies lacked adequate reporting of the timing of AEs, and there were no clear distinctions between acute or chronic AEs. Meta-analysis of the pooled data demonstrated a twofold increased risk of AEs in patients having lumbar fusion compared with patients having lumbar TDR at 2-year follow-up, and this relative risk was maintained at 5 years. Furthermore, the pooled data demonstrated a 1.7 times greater relative risk of reoperation in the fusion group compared with lumbar TDR, although this risk decreased to 1.1 at 5-year follow-up. However, given the lack of quality and consistency in the methods of recording and reporting of AEs, we are unable to make a clear recommendation of one treatment over the other. Conclusions Based on the currently available literature, lumbar TDR appears to be comparable in safety to lumbar fusion. However, due to lack of consistency in reporting of AEs, it is difficult to make conclusions regarding the true safety profile of lumbar TDR. Standardization in AE reporting will significantly improve the reliability of the current literature.

Introduction

Lumbar total disk replacement (TDR) was developed as an alternative to lumbar spinal fusion for the treatment of painful degenerative disk disease. Theoretically, lumbar TDR has the potential to decrease the impact of spinal stiffness and adjacent segment degeneration/stenosis associated with lumbar fusion.1 2 3 4 5 6 7 8 9 Lumbar TDR experienced an initial groundswell of support following its introduction, but recent studies have shown decreased enthusiasm, in part due to unpredictable results and long-term complications.10 11 12 13 14

Similar to other new medical technologies, lumbar TDR prostheses undergo an extensive Food and Drug Administration (FDA) approval process, which is intended to establish safety and efficacy of novel medical devices. Although time-consuming and costly, this process represents an important safeguard for patients. Approval for devices begins with preclinical studies evaluating mechanical function, biocompatibility, neurotoxicity, and other factors. In motion-sparing devices such as lumbar disk arthroplasty, wear properties of the bearing materials must also be tested. If these initial studies demonstrate that a device is safe for implantation in patients, clinical trials to evaluate efficacy and adverse event (AE) occurrence is performed, typically in the form of a randomized controlled trial (RCT).

As a part of the FDA's evaluation, a summary of safety and effectiveness data (SSED) is generated. This process includes a detailed analysis of all AEs in the pivotal trial, whether they appear related to the procedure or not. Given this approach, as well as the follow-up of these patients for as much as 10 years, it seems likely that the information gathered from the SSED provides an upper limit of the AE profile of the device.

Recently, concern has been raised regarding inconsistent reporting of AEs related to cervical TDR and bone morphogenic protein.15 16 Currently available peer-reviewed RCTs of lumbar TDR describe study methodology and report primary and secondary clinical outcome variables, such as pain reduction, functional improvement, health-related quality of life, the ability to work, and pain medication use. Although nearly all trials report AEs as well, the degree of standardization of AE reporting in lumbar TDR RCTs has not been assessed.

Carragee et al questioned whether current peer-reviewed publications provide adequate reporting of AEs in the use of new spinal technologies.15 This concern was based on the discovery of substantial differences between the AEs reported in SSED trials involving bone morphogenic protein and those reported in peer-reviewed literature, with important AEs having been omitted in the published studies.15 This analysis has now been verified by two further independent reviews.

As a relatively new and unproven technology, a thorough analysis of the literature can provide both patients and clinicians with an opportunity to make an informed decision regarding the risks and benefits of lumbar TDR. In this article, we will compare the AEs associated with lumbar fusion with those associated with lumbar TDR to address three goals: (1) evaluate the quality of AE reporting; (2) compare the AE risk associated with lumbar fusion versus lumbar TDR; and (3) provide recommendations on how to improve quality of reporting in peer-reviewed publications as it relates to AE.

To compare the quality of AE methodology and reporting among randomized trials comparing lumbar fusion with lumbar TDR using established AE reporting systems.

To compare the AEs and reoperations of lumbar spinal fusion with those from lumbar TDR.

To make recommendations on how to report AEs in RCTs so that surgeons and patients have more-detailed and comprehensive information when making treatment decision.

Materials and Methods

In this systematic review, PubMed, Cochrane collaboration database, and National Guideline Clearinghouse databases and bibliographies of key articles and previously published systematic reviews were searched through May 2015.

Inclusion criteria

The inclusion criteria were as follows: (1) randomized controlled trials with at least 2 years of follow-up comparing lumbar artificial disk replacement with lumbar fusion; (2) patients with axial or mechanical low back pain of ≥3 months' duration due to degenerative joint disease defined as any of the following: degenerative disk disease, facet joint disease, and spondylosis (Table 1).

Table 1

Inclusion and exclusion criteria

	Inclusion	Exclusion
Patients	• Age ≥ 18 y• Axial or mechanical LBP ≥ 3 mo due to degenerative joint disease defined as any of the following: − Degenerative disk disease − Facet joint disease − Spondylosis − Degenerative spondylolisthesis if grade 1 or less and if back pain greater than leg pain (at least 85% of patients in a study must have the diagnosis of DJD)	• <18 y old• Unspecified chronic LBP without definitive dx• Spondylolisthesis grade 2 or greater• Isthmic spondylolisthesis• Neurogenic claudication associated with stenosis• Deformity• Fracture, cancer, inflammatory disease, infection, trauma, pregnancy-related LBP• Radiculopathy• Signs of neural compression
Intervention	Lumbar:• Anterior fusion• Posterior fusion (PLF/PLIF)• Circumferential fusion	• Revision surgery
Comparator	• Lumbar TDR	• Revision surgery
Outcome	• Surgery-related adverse events• Types of complications• Reoperations• Overall adverse events• Quality of adverse event methodology• Quality of adverse event reporting	• Patient reported or clinical outcomes• Follow-up rate <70%
Study design	• RCTs with 2-y follow-up or longer	• RCT that includes subjects from another study at the same follow-up time (i.e., duplicate or overlapping studies)• Nonrandomized comparison studies• Case reports• Nonclinical studies• Case series• Cost effectiveness studies• Prognostic study

Abbreviations: DJD, degenerative joint disease; dx, diagnosis; LBP, low back pain; PLF, posterior lumbar fusion; PLIF, posterior lumbar interbody fusion; RCT, randomized controlled trial.

Exclusion criteria

The exclusion criteria were as follows: patients <18 years of age or patients with any of the following: isthmic spondylolisthesis, neurogenic claudication associated with stenosis, radiculopathy, deformity, cancer, inflammatory disease, infection, trauma, pregnancy-related low back pain.

Outcomes

Outcomes studied were: (1) overall score of the quality of AE acquisition methodology and reporting; (2) overall score of AE results reporting (the scoring systems are published elsewhere and included in the online supplementary material [supplementary Table 1]16); (3) AEs defined as those secondary to the procedure and including medical and surgery-related events; and (4) reoperations.

Analysis

Overall scores for AE acquisition quality and reporting and the risk of AEs and reoperations are reported for each study. For individual studies, we calculated relative risks (RRs) and their 95% confidence intervals (CIs) comparing lumbar TDR with fusion. To test whether the differences in risks are statistically significant, we used the chi-square test. Where possible, we combined studies and calculated pooled risks and RRs using a random-effects model to account for heterogeneity. Forest plots were used to present pooled comparisons when possible. We performed the statistical analyses using Stata 9.1 (StataCorp LP, College Station, Texas, United States).

Results

A total of 18 RCTs were identified comparing lumbar TDR with lumbar fusion (Fig. 1). Eleven of these were excluded for various reasons (see online supplementary Table 2). Seven met the inclusion criteria and formed the basis for this report (see online supplementary Table 3). Two of these studies were continuations of prior RCTs with a longer follow-up of 5 years. These are reported separately and not pooled with the original five studies.

Fig. 1

Flowchart showing results of literature search.

Clinical Question 1

We compared the quality of AE methodology and reporting among RCTs comparing lumbar fusion with lumbar TDR using established AE reporting systems.

The description of methods used to assess AEs was poor, with a mean score of 2.3 on a 5-point scale. The majority of the studies scored a 2. Only one study scored a 5 (Table 2) (see online supplementary Table 4 for itemized scores).

Table 2

Summary table of RCTs comparing fusion with lumbar TDR with risk of adverse events and quality of AE reporting

Study (year)	Devicea	Industry funded	Fusion			TDR			RR	95% CI		Methods clarity score	Reporting results score
			AEs (n)	Subjects (n)	Risk (%)	AEs (n)	Subjects (n)	Risk (%)		Lower	Upper
2-y studies
Berg (2009)	CharitéProDiscMaverick	No	15	72	20.8	14	80	17.5	1.2	0.62	2.3	5	4
Blumenthal (2005)	Charité	Yes	10	99	10.1	20	205	9.8	1.0	0.50	2.1	2	3
Delmarter (2011)	ProDisc-L	No	7	72	9.7	5	165	3.0	3.2	1.1	9.8	2	3
Zigler (2007)	ProDisc-L	Yes	5	75	6.7	4	161	2.5	2.7	0.73	9.6	2	3
Gornet (2011)	Maverick	Yes	22	172	12.8	17	405	4.2	3.0	1.7	5.6	2	4
Total/mean			59	490	12.0	60	1,016	5.9	2.0	1.4	2.9
5-y studies
Skold (2013)	CharitéProDiscMaverick	No	15	71	21.1	14	80	17.5	1.2	0.63	2.3	0	2
Zigler (2012)	ProDisc-L	Yes	8	75	10.7	5	161	3.1	3.4	1.2	10.1	3	4
Total/mean			23	146	15.8	19	241	7.9	2.0	1.1	3.5	2.3	3.3

Abbreviations: AE, adverse event; CI, confidence interval; RCT, randomized controlled trial; RR, relative risk; TDR, total disk replacement.

Charité, Depuy Spine, Raynham, Massachusetts, United States; ProDisc, Synthes Spine, West Chester, Pennsylvania, United States; Maverick, Medtronic, Memphis, Tennessee, United States.

Most studies described how and when the AEs were collected; however, few studies provided clear definitions of AEs, provided a grading of the severity of AEs, or described the methods of analysis for AEs.

The reporting of AEs and reoperations was slightly better with a mean score of 3.3 (Table 3; see online supplementary Table 5 for itemized score). All studies reported AEs and reoperations and most provided some level of categorization regarding those related to surgery.

Table 3

Summary table of RCTs comparing fusion with lumbar TDR with risk of reoperations and quality of AE reporting

Study (year)	Devicea	Industry funded	Fusion			TDR			RR	95% CI		Methods clarity score	Reporting results score
			AEs (n)	Subjects (n)	Risk (%)	AEs (n)	Subjects (n)	Risk (%)		Lower	Upper
2-y studies
Berg (2009)	CharitéProDiscMaverick	No	7	72	9.7	8	80	10.0	0.97	0.37	2.5	5	4
Blumenthal (2005)	Charité	Yes	9	99	9.1	11	205	5.4	1.7	0.72	4	2	3
Delmarter (2011)	ProDisc-L	No	6	72	8.3	4	165	2.4	3.4	1.0	11.8	2	3
Zigler (2007)	ProDisc-L	Yes	2	72	2.8	6	161	3.7	0.75	0.15	3.6	2	3
Gornet (2011)	Maverick	Yes	12	172	7.0	15	405	3.7	1.9	0.90	3.9	2	4
Total			36	487	7.4	44	1016	4.3	1.7	1.1	2.6
5-y studies
Skold (2013)	CharitéProDiscMaverick	No	7	71	9.9	8	80	10.0	0.99	0.37	2.6	0	2
Zigler (2012)	ProDisc-L	Yes	9	75	12.0	7	71	9.9	1.2	0.48	3.1	3	4
Total			16	146	11.0	15	151	9.9	1.1	0.57	2.1	2.3	3.3

Abbreviations: AE, adverse event; CI, confidence interval; RCT, randomized controlled trial; RR, relative risk; TDR, total disk replacement.

Charité, Depuy Spine, Raynham, Massachusetts, United States; ProDisc, Synthes Spine, West Chester, Pennsylvania, United States; Maverick, Medtronic, Memphis, Tennessee, United States.

Although most studies had specific follow-up times where AEs or reoperations were measured, few reported the timing of when they actually occurred so it was unclear whether they were acute or more chronic in nature.

Clinical Question 2

We compared the AEs and reoperations of lumbar spinal fusion with those from lumbar TDR.

The 2-year risk of surgery-related AEs varied among studies, ranging from 6.7 to 20.8% in the fusion group and 2.5 to 17.5% in the lumbar TDR group (Table 2). The 5-year risks were 10.7 to 21.1% and 3.1 to 17.5%, respectively (Table 2).

When combining the data in a meta-analysis, the 2-year pooled risk was 12% in the fusion group and 5.9% in the lumbar TDR group, which translates to an approximately 2 times greater risk of a surgery-related AE in the fusion group compared with the lumbar TDR group (RR = 2.0; 95% CI = 1.4 to 2.9; Table 2 and Fig. 2). The relative risk was similar for the 5-year follow-up (RR = 2.0; 95% CI = 1.1 to 3.5; Table 2 and Fig. 2).

Fig. 2

Surgery related to adverse events comparing lumbar fusion with lumbar TDR. Abbreviations: CI, confidence interval; M-H, Mantel-Haenszel; TDR, total disk replacement.

The 2-year risk of reoperation varied among studies, ranging from 2.8 to 9.7% in the fusion group and 2.4 to 10% in the lumbar TDR group (Table 3). The 5-year risks were 9.9 to 12% and 9.9 to 10.0%, respectively (Table 3).

When combining the data in a meta-analysis, the 2-year pooled risk was 7.4 in the fusion group and 4.3% in the lumbar TDR group, which translates to a 1.7 times greater risk of a reoperation in the fusion group compared with the lumbar TDR group (RR = 1.7; 95% CI = 1.1 to 2.6; Table 3 and Fig. 3). The risks for reoperation were similar between groups for the 5-year follow-up (RR = 1.1; 95% CI = 0.57 to 2.1; Table 3 and Fig. 3).

Fig. 3

Reoperations comparing lumbar fusion with lumbar TDR. Abbreviations: CI, confidence interval; M-H, Mantel-Haenszel; TDR, total disk replacement.

Clinical Question 3

We recommend the following changes to reporting AEs in RCTs so that surgeons and patients have more-detailed and comprehensive information when making treatment decisions.

A designated committee independent of the principal investigator and industry sponsor, such as a Clinical Event Committee, can improve transparency and accuracy of the information.17

Separate publication focused on detailed description of AEs in high-quality studies can provide a venue through which AEs can be separately reported and analyzed.

Simple questionnaires that are reliable and easy to use and enable documentation of type, severity, and timing of AEs can be utilized.

Guidelines from other specialties should be modified and incorporated for use in orthopedic surgery trials.

Future trials should use the evaluation systems reported in this article as a checklist for their methods and results.

A standardized reporting method should be used, such as the CONSORT statement, for reporting harm in clinical trials.18

Discussion

Our first goal was to evaluate the quality of AE reporting in lumbar fusion and lumbar TDR literature. Critical analysis of the currently available RCTs demonstrated a generally poor description of methods of assessing AEs. Although most studies did describe the method and timing of AE collection, there was a consistent lack of a clear definition or grading for these events. Furthermore, a high degree of variation in reporting of surgery-related AEs was seen in the reviewed studies. Although there was a somewhat better score of AE reporting, most studies lacked adequate reporting of the timing of AEs, and there were no clear distinctions between acute or chronic AEs.

Accepting these limitations, analysis of AEs and reoperations between lumbar spinal fusion and lumbar TDR demonstrated consistently higher risks of both for fusion as compared with lumbar TDR. A meta-analysis of the pooled data demonstrated a twofold increased risk of AEs in patients having lumbar fusion compared with patients having lumbar TDR at 2-year follow-up, and this relative risk was maintained at 5 years. Furthermore, a similar analysis of the pooled data demonstrated a 1.7 times greater RR of reoperation in the fusion group compared with lumbar TDR, although this RR decreased to 1.1 at 5-year follow-up. These findings were consistent across all RCTs reviewed and appeared to favor lumbar TDR over fusion. However, given the lack of quality and consistency in the methods of recording and reporting of AEs, we are unable to make a clear recommendation of one treatment over the other.

Given the absence of clear protocols and assessment tools for recording and reporting of AEs in clinical trials, it is perhaps not surprising that there remains a high degree of variability in the literature. Unfortunately, the most consistent approach has been the summary of SSED used by the FDA, which may include unrelated AEs, potentially leading to an overestimation of the risk associated with a given device. Other systems to report AEs recommend that events related to the disease or disease progression should be excluded from AE analysis.

Several features appear to be important in improving the consistency and effectiveness of AE reporting, including established definitions of AEs, documentation of timing method of discovery of AEs, and appropriate grading of the severity of each complication. This review demonstrates that even clinical studies of the highest level of medical evidence failed to provide this information. Thus, despite the strength of RCTs with regard to reducing the impact of confounding variables associated with patient diversity, surgical intervention, and surgeon related issues, we are unable to comment on this important aspect of comparative safety of these two surgical approaches to lumbar degenerative disk disease.

Although multiple studies have alluded to the fact that retrospective collection of data associated with morbidity and mortality significantly underestimates the risk of complications associated with complex spine surgery,19 20 substantial efforts at improving this situation are underway. Street et al proposed the Spine Adverse Event Severity (SAVES) system to document the occurrence of an AE at the time of surgery, at discharge, and later.19 This simple questionnaire requires minimal effort to complete, has good reliability, and is available in paper and electronic formats. Additionally, this questionnaire enables documentation of the type, severity, and timing of AEs and their effect on patients' length of stay. We recommend using the systems published in this article and the article by Anderson as a checklist when planning the methods and reporting of AEs alongside clinical results.16

A more-standardized approach has been outlined by Ioannidis, adding to the well-accepted CONSORT method of reporting RCTs.18 In this method, a systematic approach to reporting the methods used to capture AEs and the best reporting practices are outlined. Using this method as a standard, almost all studies even in highest-impact journals have been shown to have inadequately reported and documented AEs.21 22

Additionally, modification and incorporation of the recommendations from other specialties may prove to be beneficial in establishing more uniform guidelines for orthopedic surgery trials. The National Cancer Institute and the World Health Organization have established guidelines to assist in reporting AEs in oncological clinical trials; these recommendations could potentially be modified for use in orthopedic and spine surgical trials.23 24

We conclude that the lack of standardized definitions, absence of classification and collection protocols, and inconsistent reporting methodologies have led to a significant variation in AE recording and reporting in the peer-reviewed literature related to lumbar TDR. This deficiency creates a significant gap in the utility of clinical data, even among studies conducted using the highest level of sophistication from a medical evidence viewpoint. Efforts to standardize assessment and reporting of AEs in surgical trials appear warranted.

Summary and Conclusion

In short-term follow-up, there is a higher risk of AEs and reoperation in lumbar fusion compared with lumbar TDR, although differences in reoperation risks appear to diminish at longer follow-up. However, the lack of consistency in published RCTs prevents strong recommendations based on differences in AEs relating to these two surgical options. Because peer-reviewed publications lack adequate consistency in AE reporting, the impact of the conclusions presented in those reports is diminished. Going forward, improvements in methods to collect, assess, and report AEs are needed.