Is There an Association Between Authors' Conflicts of Interest and Outcomes in Clinical Studies Involving Autologous Chondrocyte Implantation?

BACKGROUND: Autologous chondrocyte implantation (ACI) is an increasingly popular technique for the treatment of articular cartilage defects. Because several companies have financial interests in ACI, it is important to consider possible conflicts of interest when evaluating studies reporting outcomes of ACI.
PURPOSE: To determine whether there is an association between authors' financial conflicts of interest and the outcomes of ACI studies. STUDY
DESIGN: Cross-sectional study.
METHODS: A search of PubMed and MEDLINE databases for "autologous chondrocyte implantation" was performed. Clinical studies published after 2012 through May 15, 2019, and in English were included. Studies were determined to have financial conflicts of interest if any contributing author had relevant conflicts, either self-reported in the published study's disclosures section or reported online in the American Academy of Orthopaedic Surgeons Disclosure database or the Centers for Medicare & Medicaid Services Open Payments database. The outcomes of each study were rated as favorable, equivocal, or unfavorable based on predefined criteria and then tested for association with conflicts of interest through use of the Fisher exact test.
RESULTS: A total of 79 studies met the inclusion criteria. Nearly all studies were of level 3 or 4 evidence. Conflicts of interest were established in 51.90% of studies (n = 41). Conflicts that were not self-reported by the authors were discovered in 18% of studies. The level of evidence was not associated with conflict of interest. No statistically significant difference was found in the rate of favorable outcomes between studies with conflicts (92.68%) and those with no conflicts (81.58%) (P = .126). Publications by US authors were more likely to have financial conflicts of interest (P = .003).
CONCLUSION: Favorable results were reported in a majority of studies involving ACI. No statistical association was found between the frequency of favorable outcomes and the presence of financial conflicts of interest, country of authorship, or level of evidence. There was a trend toward more favorable outcomes in studies with conflicts of interest. Additionally, nearly 20% of publications had possible conflicts found online that were not self-reported. It is critical for orthopaedic surgeons to judiciously evaluate published studies and consider financial conflicts of interest before performing ACI techniques on patients.

Orthopaedic surgery is one of the most rapidly developing fields in medicine.[8,31] The development of new procedures and operative concepts necessitates an emphasis on quality research in the field to provide strong, evidence-based care. However, the cost of performing high-quality, prospective studies to assess new devices and techniques can be steep. Although companies often fund studies to assist in the safe and effective development of their products, care must be taken to ensure impartial data. With respect to biomedical research, the industry’s portion of total investment nearly doubled from 32% in 1980 to 62% in 2000, with a corresponding decrease in the United States federal government’s portion.[4]

The treatment of articular cartilage defects is a particular area of orthopaedic research that has garnered increased attention.[5,11,21] Multiple techniques have been proposed to treat focal chondral defects, with 2 of the most recent being autologous chondrocyte implantation (ACI) and matrix-induced ACI (MACI).[5,11] ACI/MACI involves 2 stages. During the first, chondrocytes are harvested arthroscopically from areas of decreased weightbearing. Then, in the second stage, these cells are expanded in vitro and later reimplanted into the defect roughly 1 to 2 months after the initial procedure.[17] Unlike many other cartilage treatment options, ACI/MACI relies on patented techniques and products that make it the most expensive surgical option in the treatment of articular cartilage defects.[1,27] As such, performing research on patients can be costly.

A number of authors have discussed conflicts of interest in medicine, including the field of orthopaedics.[7,23,24,32] Friedman and Richter[7] found a strong association between studies whose authors had conflicts of interest and also reported positive findings, including studies in well-known journals such as the New England Journal of Medicine and the Journal of the American Medical Association. Within orthopaedics, Zuckerman et al[32] found that the number of American Academy of Orthopaedic Surgeons (AAOS) presenters who reported conflicts of interest dramatically increased from 10% in 1985 to 74% in 2002.

As relationships among industry, institutions, and physicians are intensifying, it is important to consider author conflicts of interest when evaluating these studies. To our knowledge, conflict of interest has not been assessed with regard to ACI outcomes. As such, the purpose of this study is to determine the relationship between the presence of financial conflicts of interest of study authors and published outcomes of ACI studies.

Methods

A search of the PubMed and MEDLINE databases for “autologous chondrocyte implantation” was performed on May 15, 2019. The search included studies published in English, from 2013 to the date of search, that reported clinical outcomes of patients who underwent ACI or MACI procedures. Clinical outcomes included patient-reported outcomes, radiographic or other imaging findings, and cost. Exclusion criteria included biomechanical, bench, or cadaveric studies; commentaries, editorials, or case reports; and studies with <20 patients. Articles without a disclosures section or conflict of interest statement available either in the printed journal or on the corresponding website were excluded from analysis. Additionally, we excluded studies that examined cartilage procedures broadly but did not separate ACI/MACI procedures from other cartilage techniques. The level of evidence for each included study was determined by guidelines proposed by Sackett.[26] Studies were then categorized by country of origin based on the location listed for the primary author. The anatomic region of each study was also recorded.

Determining Conflict of Interest Status

To determine conflict of interest status, we searched for all authors from each study in both the AAOS Disclosure database (http://disclosure.aaos.org) and the Open Payments database (managed by the US Centers for Medicare & Medicaid Services [CMS]) (http://openpaymentsdata.cms.gov).[2,25] Studies were determined to have financial conflicts of interest if any of the study authors met any of the three following criteria: (1) self-reported conflicts as listed in the published article’s disclosures section in print or online, including industry funding of the study; (2) relevant disclosures on the online AAOS Disclosure database from the year of publication or prior; and (3) relevant conflicts on the online Open Payments database from the year of study publication or prior. For studies with no authors from the United States, conflict of interest status was determined based on only self-reported conflicts and AAOS disclosures.

Relevant conflicts were identified as license or royalty fees, consulting fees, advisory position/speaker, employee, stock options, or research funding from any company that produces ACI or a similar cartilage treatment option. Because it is impossible to determine whether a conflict of interest is directly related to ACI, we used this more inclusive method, which has been described in prior published studies.[7,23] If one or more authors had relevant conflicts, the study was considered to entail a conflict. Funding from nonindustry foundations, such as national research organizations, was not considered a conflict.

Determination of Outcome

The outcomes of each study were rated as favorable, equivocal, or unfavorable according to predefined criteria based on previously published protocols.[7,23] Favorable outcomes were those that showed equivalence or superiority (P < .05) of ACI to a currently accepted treatment or showed no difference with regard to complications or adverse events (P > .05). Studies in which a favorable conclusion was reached were also determined to be favorable, as well as those in which the discussion and conclusion claimed ACI as superior or equivalent to current treatments if no control was used. Studies with unfavorable outcomes were those in which there were no reported clinical benefits (P > .05) or in which ACI resulted in inferior patient-reported outcomes or a higher rate of complications compared with a control. Studies with equivocal results were those that reported a statistically significant clinical benefit with a high complication rate or those with inconclusive results. A higher complication was defined as a statistically higher complication rate than a comparison (P < .05) or a reported higher complication rate than previously published literature if it was mentioned in the discussion section of that paper. Each study was assessed by 2 reviewers (B.C.M., A.B.) blinded to the study funding and the other rater’s determination. A kappa statistic was calculated to determine level of agreement between reviewers. Studies in which there was disagreement between reviewers were discussed until a final determination was made.

Statistical Analysis

Statistical analysis was performed using Stata/MP 13.1 for Mac (StataCorp). Descriptive statistics are reported as percentages. Outcomes and levels of evidence were compared between studies with and without conflicts of interest and were tested for association using the Fisher exact test. Additionally, studies performed in the United States and those performed elsewhere were compared regarding outcomes as well as the presence of financial conflicts of interest. Last, the level of evidence was tested for association with outcome. An additional subanalysis of studies from the United States was performed to assess outcomes between studies with and without conflicts of interest. P < .05 was considered statistically significant.

A post hoc power analysis was performed for a chi-square analysis using the proportions and sample sizes for the outcomes of studies with and without conflicts of interest, with an alpha set at .05. The calculation determined that the power of our analysis was 0.263, indicating that the present analysis was underpowered.

Results

The initial search resulted in 670 unique studies. After title and abstract screening and full-text review, a total of 79 studies met final inclusion criteria and were included in the analysis (Figure 1).

Figure 1.

Flowchart depicting inclusion and exclusion criteria.

Nearly all studies were of level 3 (27.85%) or level 4 (65.82%) evidence. A large majority (70.89%) had a primary research site outside of the United States. The knee was the most common surgical location reported (84.81%). A complete summary of included studies can be found in Table 1. Conflicts of interest were established in 51.90% of studies (n = 41). Table 2 summarizes conflicts of interest status in further detail.

TABLE 1

Summary of Included Studies (N = 79)

Characteristic	n (%)
Level of evidencea
1	2 (2.53)
2	3 (3.80)
3	22 (27.85)
4	52 (65.82)
Country of originb
United States	23 (29.11)
Other	56 (70.89)
Anatomic location
Knee	67 (84.81)
Ankle	10 (12.66)
Hip	2 (2.53)
Year of publication
2013	11 (13.92)
2014	18 (22.78)
2015	8 (10.13)
2016	13 (16.46)
2017	13 (16.46)
2018	10 (12.66)
2019	6 (7.59)

Level of evidence according to Sackett.[26]

Studies with any author with listed US affiliation.

TABLE 2

Conflicts of Interest

	n (%)
Self-reported conflicts according to the study’s disclosure section
No conflicts	52 (65.82)
Conflicts	27 (34.18)
Online conflicts according to the AAOS Disclosure databaseb
No conflicts	65 (82.28)
Conflicts	14 (17.72)
Online conflicts according to the Open Payments databaseb
No conflicts	7 (30.43)
Conflicts	16 (69.57)
Conflict of interest category
No conflicts by any source	38 (48.10)
Conflicts by self-reporting, not by online databases	12 (15.19)
Conflicts by online databases, not by self-reporting	14 (17.72)
Conflicts by self-reporting and online databases	15 (18.99)

AAOS, American Academy of Orthopaedic Surgeons.

Disclosures from the year of study publication or prior. The Open Payments database is applicable to US authors only.

Level of evidence was not associated with conflict of interest (P = .852). We did not identify a statistically significant difference in the rate of favorable outcomes between studies with conflicts of interest (92.68%) and those without (81.58%) (P = .126) (Table 3). Publications with authors who had US affiliations were more likely to have financial conflicts of interest (P = .003) (Table 4). There was 91.14% agreement between reviewers for determining an outcome as favorable, equivocal, or unfavorable, with a kappa value of 0.594 (P < .001). We found a slightly higher rate of favorable outcomes in studies with lower level of evidence, but this association did not reach statistical significance (P = .157) (Table 5). A complete summary of included studies can be found in Appendix Table A1.

TABLE 3

Level of Evidence and Outcome by Conflict of Interest

	No Conflicts (n = 41)	Conflicts (n = 38)	P Valueb
Level of evidencec			.852
1 or 2	2 (4.88)	3 (7.89)
3	6 (14.63)	5 (13.16)
4	33 (80.49)	30 (78.95)
Outcome			.126
Favorable	38 (92.68)	31 (81.58)
Equivocal/unfavorable	3 (7.32)	7 (18.42)

Data are expressed as n (%).

Fisher exact test for categorical variables.

Level of evidence according to Sackett.[26]

TABLE 4

Study Outcome by Country of Author Affiliation

	United States (n = 23)	Other (n = 56)	P Valueb
Outcome			.395
Favorable	21 (91.30)	48 (85.71)
Equivocal/unfavorable	2 (8.70)	8 (14.29)
Presence of financial conflicts			.003
Conflicts	18 (78.26)	23 (41.07)
No conflicts	5 (21.74)	33 (58.93)

Data are expressed as n (%).

value calculated using Fisher exact test for categorical variables. Bolded P value indicates statistically significant between-group difference (P < .05).

TABLE 5

Outcome by Level of Evidence

	Favorable (n = 69)	Equivocal/Unfavorable (n = 10)	P Valueb
Level of evidencec			.157
1 or 2	3 (4.35)	2 (20.00)
3	10 (14.49)	1 (10.00)
4	56 (81.16)	7 (70.00)

Data are expressed as n (%).

value calculated using Fisher exact test for categorical variables.

Level of evidence according to Sackett.[26]

TABLE A1

Summary of Included Studies

Title	First Author	Country	LOEa	Outcome	Conflict of Interest
A comparison of the responsiveness of 4 commonly used patient-reported outcome instruments at 5 years after matrix-induced autologous chondrocyte implantation.	Ebert	Non-US	4	Favorable	No conflicts
A prospective clinical and radiological evaluation at 5 years after arthroscopic matrix-induced autologous chondrocyte implantation.	Ebert	Non-US	4	Favorable	Conflicts
A randomized multicenter trial comparing autologous chondrocyte implantation with microfracture: long-term follow-up at 14 to 15 years.	Knutsen	Non-US	1	Equivocal	Conflicts
Accelerated weightbearing rehabilitation after matrix-induced autologous chondrocyte implantation in the tibiofemoral joint: early clinical and radiological outcomes.	Edwards	Non-US	2	Favorable	No conflicts
Analysis of the autologous chondrocyte quality of matrix-based autologous chondrocyte implantation in the knee joint.	Niethammer	Non-US	4	Favorable	No conflicts
Arthroscopic autologous chondrocyte implantation in the hip for the treatment of full-thickness cartilage defects—a case series of 29 patients and review of the literature.	Thier	US	4	Favorable	No conflicts
Arthroscopic treatment of osteochondral lesions of the talus using juvenile articular cartilage allograft and autologous bone marrow aspirate concentration.	DeSandis	US	4	Favorable	No conflicts
Arthroscopic treatment of patellar and trochlear cartilage lesions with matrix encapsulated chondrocyte implantation versus microfracture: quantitative assessment with MRI T2-mapping and MOCART at 4-year follow-up.	Olivos	Non-US	3	Favorable	No conflicts
Arthroscopic versus open matrix-induced autologous chondrocyte implantation: results and implications for rehabilitation.	Edwards	Non-US	4	Favorable	Conflicts
Autologous bone plug supplemented with autologous chondrocyte implantation in osteochondral defects of the knee.	Bhattacharjee	Non-US	4	Favorable	Conflicts
Autologous chondrocyte implantation and anteromedialization for isolated patellar articular cartilage lesions: 5- to 11-year follow-up.	Gillogly	US	4	Favorable	Conflicts
Autologous chondrocyte implantation and tibial tubercle osteotomy for patellofemoral chondral defects: improved pain relief and occupational outcomes among US Army servicemembers.	Zarkadis	US	4	Favorable	Conflicts
Autologous chondrocyte implantation for bipolar chondral lesions in the patellofemoral compartment: clinical outcomes at a mean 9 years’ follow-up.	Ogura	US	4	Favorable	Conflicts
Autologous chondrocyte implantation for talar osteochondral lesions: comparison between 5-year follow-up magnetic resonance imaging findings and 7-year follow-up clinical results.	Pagliazzi	Non-US	4	Favorable	No conflicts
Autologous chondrocyte implantation improves knee-specific functional outcomes and health-related quality of life in adolescent patients.	Cvetanovich	US	4	Favorable	Conflicts
Autologous chondrocyte implantation in the knee: mid-term to long-term results.	Nawaz	Non-US	4	Favorable	Conflicts
Autologous chondrocyte implantation in the patella: a multicenter experience.	Gomoll	US	4	Favorable	Conflicts
Autologous chondrocyte implantation of the ankle: 2- to 10-year results.	Kwak	US	4	Favorable	Conflicts
Autologous chondrocyte implantation to isolated patella cartilage defects.	von Keudell	US	4	Favorable	Conflicts
Biological knee reconstruction with concomitant autologous chondrocyte implantation and meniscal allograft transplantation: mid- to long-term outcomes.	Ogura	US	4	Favorable	Conflicts
Cartilage defect treatment using high-density autologous chondrocyte implantation: two-year follow-up.	Lopez-Alcorocho	Non-US	3	Favorable	No conflicts
Cartilage repair procedures associated with high tibial osteotomy in varus knees: clinical results at 11 years’ follow-up.	Ferruzzi	Non-US	4	Equivocal	No conflicts
Cartilage repair surgery prevents progression of knee degeneration.	Jungmann	US	3	Favorable	Conflicts
Cell-seeded autologous chondrocyte implantation: a simplified implantation technique that maintains high clinical outcomes.	Gomoll	US	4	Favorable	Conflicts
Clinical and radiographical ten years long-term outcome of microfracture vs. autologous chondrocyte implantation: a matched-pair analysis.	Ossendorff	Non-US	4	Favorable	No conflicts
Clinical outcomes after cell-seeded autologous chondrocyte implantation of the knee: when can success or failure be predicted?	Pestka	Non-US	3	Favorable	No conflicts
Clinical outcomes and survival rate of autologous chondrocyte implantation with and without concomitant meniscus allograft transplantation: 10- to 15-year follow-up study.	Yoon	Non-US	4	Favorable	No conflicts
Clinical profiling in cartilage regeneration: prognostic factors for midterm results of matrix-assisted autologous chondrocyte transplantation.	Filardo	Non-US	4	Favorable	Conflicts
Comparison of juvenile allogenous articular cartilage and bone marrow aspirate concentrate versus microfracture with and without bone marrow aspirate concentrate in arthroscopic treatment of talar osteochondral lesions.	Karnovsky	US	4	Equivocal	No conflicts
Correlation of MRI appearance of autologous chondrocyte implantation in the ankle with clinical outcome.	Chan	US	4	Favorable	Conflicts
Development of a tool to predict outcome of autologous chondrocyte implantation.	Dugard	Non-US	4	Favorable	No conflicts
Factors influencing the results in matrix-associated autologous chondrocyte implantation: a 2 - 5 year follow-up study.	Gursoy	Non-US	4	Favorable	No conflicts
Factors predictive of outcome 5 years after matrix-induced autologous chondrocyte implantation in the tibiofemoral joint.	Ebert	Non-US	4	Favorable	No conflicts
First-generation autologous chondrocyte implantation in patients with cartilage defects of the knee: 7 to 14 years’ clinical and magnetic resonance imaging follow-up evaluation.	Niemeyer	Non-US	4	Favorable	Conflicts
Follow-up of a new arthroscopic technique for implantation of matrix-encapsulated autologous chondrocytes in the knee.	Ibarra	Non-US	4	Favorable	No conflicts
Gel-type autologous chondrocyte implantation for cartilage repair in patients with prior ACL reconstruction: a retrospective two year follow-up.	van Duijvenbode	Non-US	4	Favorable	No conflicts
Good clinical and MRI outcome after arthroscopic autologous chondrocyte implantation for cartilage repair in the knee.	Siebold	Non-US	4	Favorable	Conflicts
Graft hypertrophy of matrix-based autologous chondrocyte implantation: a two-year follow-up study of NOVOCART 3D implantation in the knee.	Niethammer	Non-US	4	Equivocal	No conflicts
High-density autologous chondrocyte implantation as treatment for ankle osteochondral defects.	Lopez-Alcorocho	Non-US	4	Favorable	No conflicts
Incidence, degree, and development of graft hypertrophy 24 months after matrix-induced autologous chondrocyte implantation: association with clinical outcomes.	Ebert	Non-US	4	Favorable	Conflicts
Incomplete defect filling after third generation autologous chondrocyte implantation.	Niethammer	Non-US	4	Equivocal	No conflicts
Influence of sex on the outcome of autologous chondrocyte implantation in chondral defects of the knee.	Kreuz	Non-US	4	Favorable	No conflicts
Injectable autologous chondrocyte implantation (ACI) in acetabular cartilage defects---three-year results.	Krueger	Non-US	4	Favorable	Conflicts
Intermediate- to long-term results of combined anterior cruciate ligament reconstruction and autologous chondrocyte implantation.	Pike	US	4	Favorable	Conflicts
Is the transplant quality at the time of surgery adequate for matrix-guided autologous cartilage transplantation? A pilot study.	Zellner	Non-US	4	Favorable	No conflicts
Long-term clinical and MRI results of matrix-assisted autologous chondrocyte implantation for articular cartilage defects of the knee.	Kreuz	Non-US	4	Favorable	Conflicts
Long-term clinical results and MRI changes after autologous chondrocyte implantation in the knee of young and active middle aged patients.	Rosa	Non-US	4	Favorable	No conflicts
Long-term follow-up evaluation of autologous chondrocyte implantation for symptomatic cartilage lesions of the knee: a single-centre prospective study.	Berruto	Non-US	3	Favorable	No conflicts
Long-term outcomes after first-generation autologous chondrocyte implantation for cartilage defects of the knee.	Niemeyer	Non-US	4	Favorable	Conflicts
Long-term outcomes of autologous chondrocyte implantation in adolescent patients.	Ogura	US	4	Favorable	Conflicts
Long-term results of autologous chondrocyte implantation in the knee for chronic chondral and osteochondral defects.	Biant	Non-US	4	Favorable	No conflicts
Magnetic resonance imaging parameters at 1 year correlate with clinical outcomes up to 17 years after autologous chondrocyte implantation.	McCarthy	Non-US	3	Equivocal	Conflicts
Matrix based autologous chondrocyte implantation in children and adolescents: a match paired analysis in a follow-up over three years post-operation.	Niethammer	Non-US	4	Favorable	No conflicts
Matrix-associated autologous chondrocyte implantation is an effective treatment at midterm follow-up in adolescents and young adults.	Hoburg	Non-US	4	Favorable	Conflicts
Matrix-associated autologous chondrocyte implantation: a clinical follow-up at 15 years.	Gille	Non-US	4	Favorable	No conflicts
Matrix-induced autologous chondrocyte implantation (MACI) for chondral defects in the patellofemoral joint.	Meyerkort	Non-US	4	Favorable	Conflicts
Matrix-induced autologous chondrocyte implantation (MACI) in the knee: clinical outcomes and challenges.	Basad	Non-US	4	Favorable	Conflicts
Matrix-induced autologous chondrocyte implantation for the treatment of chondral defects of the knees in Chinese patients.	Zhang	Non-US	4	Favorable	Conflicts
Matrix-induced autologous chondrocyte implantation versus multipotent stem cells for the treatment of large patellofemoral chondral lesions: a nonrandomized prospective trial.	Gobbi	Non-US	3	Favorable	Conflicts
Matrix-induced autologous mesenchymal stem cell implantation versus matrix-induced autologous chondrocyte implantation in the treatment of chondral defects of the knee: a 2-year randomized study.	Akgun	US	2	Favorable	No conflicts
Minimal clinically important differences and substantial clinical benefit in patient-reported outcome measures after autologous chondrocyte implantation.	Ogura	US	4	Favorable	Conflicts
Nasal chondrocyte-based engineered autologous cartilage tissue for repair of articular cartilage defects: an observational first-in-human trial.	Mumme	Non-US	4	Favorable	Conflicts
Osteochondral scaffold reconstruction for complex knee lesions: a comparative evaluation.	Filardo	Non-US		Favorable	Conflicts
Outcome of combined autologous chondrocyte implantation and anterior cruciate ligament reconstruction.	Dhinsa	Non-US	4	Favorable	No conflicts
Patient-oriented and performance-based outcomes after knee autologous chondrocyte implantation: a timeline for the first year of recovery.	Howard	US	4	Favorable	Conflicts
Prospective clinical and radiologic evaluation of patellofemoral matrix-induced autologous chondrocyte implantation.	Ebert	Non-US	4	Favorable	No conflicts
Reconstruction of osteochondral lesions of the talus with autologous spongiosa grafts and autologous matrix-induced chondrogenesis.	Valderrabano	US	4	Favorable	Conflicts
Regenerative treatment in osteochondral lesions of the talus: autologous chondrocyte implantation versus one-step bone marrow derived cells transplantation.	Buda	Non-US	3	Favorable	No conflicts
Repair potential of matrix-induced bone marrow aspirate concentrate and matrix-induced autologous chondrocyte implantation for talar osteochondral repair: patterns of some catabolic, inflammatory, and pain mediators.	Desando	Non-US	3	Favorable	Conflicts
Return to preoperative function after autologous cartilage implantation of the knee in active military servicemembers.	Zarkadis	US	4	Equivocal	Conflicts
Return to sports activity and work after autologous chondrocyte implantation of the knee: which factors influence outcomes?	Pestka	Non-US	4	Equivocal	No conflicts
Revision cartilage cell transplantation for failed autologous chondrocyte transplantation in chronic osteochondral defects of the knee.	Vijayan	Non-US	4	Favorable	No conflicts
Revision surgery after cartilage repair: data from the German Cartilage Registry (KnorpelRegister DGOU).	Pestka	Non-US	3	Favorable	Conflicts
Seven-year follow-up of matrix-induced autologous implantation in talus articular defects.	Kreulen	US	4	Favorable	No conflicts
The effect of cell dose on the early magnetic resonance morphological outcomes of autologous cell implantation for articular cartilage defects in the knee: a randomized clinical trial.	Niemeyer	Non-US	2	Favorable	Conflicts
The John Insall Award: a minimum 10-year outcome study of autologous chondrocyte implantation.	Minas	US	4	Favorable	Conflicts
The progression of isokinetic knee strength after matrix-induced autologous chondrocyte implantation: implications for rehabilitation and return to activity.	Ebert	Non-US	4	Equivocal	No conflicts
Third-generation autologous chondrocyte implantation versus mosaicplasty for knee cartilage injury: 2-year randomized trial.	Clave	Non-US	1	Unfavorable	No conflicts
Treatment of full-thickness chondral defects with hyalograft C in the knee: long-term results.	Brix	Non-US	4	Favorable	No conflicts

Level of evidence (LOE) according to Sackett.[26]

We analyzed 23 US-based studies, 18 of which (78.3%) had conflicts of interest. Further, 17.39% of these had conflicts by online reporting that were not disclosed in the published article (Table 6). Among US-based studies with conflicts, 94.4% were found to have favorable outcomes, whereas 80.0% of studies without conflicts of interest had favorable outcomes (P = .395) (Table 7).

TABLE 6

Conflicts of Interest for Studies Performed in the United States (n = 23)

	n (%)
Self-reported conflicts according to the study’s disclosure section
No conflicts	9 (39.13)
Conflicts	14 (60.97)
Online conflicts according to the AAOS Disclosure databaseb
No conflicts	22 (95.65)
Conflicts	1 (4.35)
Online conflicts according to the Open Payments databaseb
No conflicts	7 (30.43)
Conflicts	16 (69.57)
Conflict of interest category
No conflicts by any source	5 (21.47)
Conflicts by self-reporting, not by online databases	2 (8.70)
Conflicts by online databases, not by self-reporting	4 (17.34)
Conflicts by self-reporting and online databases	12 (52.17)

AAOS, American Academy of Orthopaedic Surgeons.

Disclosures from the year of study publication or prior. The Open Payments database is applicable to US authors only.

TABLE 7

Outcome by Conflict of Interest for Studies Performed in the United States

	Conflicts (n = 18)	No Conflicts (n = 5)	P Valueb
Outcome			.395
Favorable	17 (94.4)	4 (80.0)
Equivocal/unfavorable	1 (5.6)	1 (20.0)

Data are expressed as n (%).

value calculated using Fisher exact test for categorical variables.

Discussion

ACI is rapidly gaining favor in the treatment of focal articular cartilage defects, as this therapy has demonstrated promising clinical outcomes compared with options such as microfracture and osteochondral autograft transplant. However, questions have arisen regarding the true benefit as well as cost-effectiveness of ACI procedures.[27] As orthopaedic surgeons bring proposed procedures into their practices, trustworthy, evidence-based clinical research support is critical. A number of clinical research studies are led by authors with significant industry support, and it is important to consider the effects of possible bias when evaluating literature before implementing new techniques into practice. The results of this study demonstrate that a majority of studies regarding ACI report overall positive outcomes. Notably, there was no statistically significant difference in the rate of favorable outcomes between studies with and without conflicts of interest. Publications including authors with US affiliations were more likely to have financial conflicts of interest.

Nearly 95% of the included studies were level 3 or level 4 evidence, predominantly retrospective cohort studies or case series, and there were only two level 1 studies. This is in sharp contrast to a study evaluating conflicts of interest in the literature on cervical spine disk arthroplasty,[23] in which 50% of included studies were level 1 or 2. The lack of a high level of evidence in ACI literature may be explained by the high cost of ACI, which may prohibit running a randomized controlled trial without financial support. Additionally, surgeons may be unwilling to randomize their own patients to a treatment they believe may be inferior to ACI for certain pathologies. Last, publication bias may contribute to the overall few numbers of nonfavorable outcomes. The effect of publication bias in clinical studies has been well-described, as studies with nonsignificant results are more than twice as likely to not be published than studies with significant results.[10,12,13,15,22,28,29] Although the reasons for this can only be theorized, possible explanations may be that investigators who were eager to find a significant difference do not want to publish nonsignificant results, journal editors and reviewers are less likely to publish results without a significant impact on practice, or industry funders are hesitant to publish randomized controlled trials that do not demonstrate positive outcomes. The currently accepted use of ACI is in patients with large, full-thickness chondral defects, which may not be well-treated with other cartilage techniques, thus limiting the clinical appropriateness of a randomized trial.

Of the studies included in this data set, 34% self-reported their conflicts of interest in the published article’s disclosure section. Although a majority of authors with identifiable conflicts also self-reported, there were several discrepancies. Approximately 15% of included studies that had conflicts by self-reported disclosure did not have identifiable conflicts online, which may be due to the percentage of European literature that is not required to report to the CMS database or the AAOS unless presented. Conversely, 18% of included studies had conflicts that were revealed by online reporting, not by self-reported disclosures. When only studies performed in the United States were analyzed, a similar rate of underreporting was identified. If the high level of discrepancy is due to lack of self-reporting, journals may need to revamp their policies about confirming author conflict status. Regardless of the reason for the high level of unreported conflicts, it is concerning that the level of necessary detail is not available for readers to distinguish how relevant a conflict may be. Further regulations about the level of detail required in the CMS database should be considered to mitigate this issue.

Discrepancies in conflict of interest reporting have been detailed in the literature, with Hannon et al[9] finding a 35% rate of inconsistency between physician and industry-reported financial relationships for the nearly 2000 presenters at the 2014 AAOS annual meeting. Similarly, in an analysis of popular orthopaedic journals, Janney et al[14] noted that 13% of first and/or last authors had disclosure discrepancies. A strength of the current study is the use of both the AAOS Disclosure database and the Open Payments database in the determination of conflict of interest status. However, due to limitations in details of online reporting, it is possible that the conflicts noted online may not be truly related to specific publications, thus overestimating the true rate of conflicts. Regardless, to limit subjectivity into the study method, all possible related conflicts were included. One could argue that any investment or relationship in a particular company could benefit from any area of said company having positive outcomes.

In recent years, a focus on conflict of interest reporting has increased dramatically in both medical and orthopaedic literature.[18,20,32] A majority of this literature has found positive associations between the presence of financial conflicts of interest and positive study outcomes.[7,16,19,23,24] In a 2003 study assessing the association between conflicts of interest and outcomes in multiple orthopaedic journals, Leopold et al[19] noted a 79% rate of positive outcomes in studies with financial conflicts versus a 63% rate in studies that received no industry funding, a statistically significant difference. The results of the current study were not quite as drastic. Although almost 93% of the studies with conflicts of interest reported favorable outcomes, compared with 82% of studies that did not have conflicts of interest, this result was not statistically significant. The subanalysis of studies performed in the United States showed similar percentages of favorable outcomes as in the primary analysis of all study locations. Studies in the spine literature have reported similar findings of a high rate of favorable outcomes.[3,30] Given the 11% difference in positive outcomes, it is possible that the present study was underpowered to detect a difference. Similarly, publication bias toward positive outcomes may have suppressed the number of available studies with unfavorable outcomes.[6]

When we assessed country of origin, 78% of US-based studies had conflicts of interest compared with 41% of non-US studies. This is perhaps explained by the fact that the CMS is a federal agency within the US government, and the mandated reporting process may result in a truer percentage of studies with conflicts of interest. Similarly, socioeconomic variations may play a role; many of the non-US countries have socialized health care, and as such, that research may be less influenced by funding from industrial sponsors and more likely to be funded by national health care organizations. Last, the indications and patient selection for performing ACI may vary in different regions and countries, introducing selection bias to the results. Regardless of the reason for the discrepancy in percentage of conflicts, there was no association between country of authorship and favorable outcomes. This supports what has been reported in prior orthopaedic literature, as Leopold et al[19] found that author country of origin was not related to an association between conflict of interest and positive study outcomes.

The present study is not without limitations. First, the present data included only 2 level 1 studies; this is worth noting because level 1 studies represent the gold standard of clinical research, which theoretically should be the most resistant to industry bias. However, this may be related to the overall lack of level 1 studies published in the literature rather than a lack of level 1 studies that met our inclusion criteria. Additionally, there is a general lack of specificity in conflict of interest data available online, which could diminish the accuracy of our data set. However, given that companies are expanding their treatment offerings across respective fields, one may argue that even if a certain author’s conflict did not directly relate to the study matter at hand, that author may still benefit if the company performs well in a related area. Another limitation is the lack of CMS reporting for European authors, as previously mentioned. Our inclusion of multiple online disclosure sources may help offset inaccuracy concerns from lack of reporting. However, many non-US authors do not participate in AAOS events and would not be captured by the AAOS database or CMS reporting. Despite this, many non-US authors had conflicts listed on the AAOS disclosure database. Additionally, due to the limited total number of studies involving ACI, a comparative analysis of other cartilage procedures could not be performed, as many of the studies included were case series. Future research assessing these comparisons, as well as the rate of conflicts in other orthopaedic literature when compared with ACI, will be beneficial to orthopaedic surgeons when evaluating studies going forward. Last, the grading of study outcomes in this paper had areas of subjectivity. However, each reviewer was blinded to both study funding status and the other reviewer’s assessments, and the reviewers demonstrated high intraobserver reliability. Furthermore, differing determinations were jointly reviewed until mutual conclusions were drawn.

The results of this study suggest that favorable results are reported in a majority of studies involving ACI. No statistical association was found between the frequency of favorable outcomes and the presence of financial conflicts of interest, country of authorship, or level of evidence. We found a 11% difference in favorable outcomes between studies with and without conflicts of interest, which did not reach statistical significance. This may be due to the fact that the study was underpowered, as demonstrated by the post hoc power analysis. Additionally, nearly 20% of papers had possible conflicts found online that were not reported in the published study. It is critical for orthopaedic surgeons to judiciously evaluate published studies and consider financial conflicts of interest before performing ACI techniques on patients.