Abstracts Accepted for Presentation at Orthopaedic Trauma or Arthroplasty Conferences: Which Conference Is the Best Indicator of Future Publication?

Time and financial resources pose limitations to orthopaedic surgeons wishing to advance their orthopaedic knowledge, and surgeons frequently must choose one meeting to attend. We sought to determine whether abstracts presented at the American Academy of Orthopaedic Surgeons (AAOS) annual meeting or the trauma (Orthopaedic Trauma Association [OTA]) or arthroplasty (American Association of Hip and Knee Surgeons [AAHKS]) subspecialty meetings, respectively, were higher yield with respect to material ultimately being published. We hypothesized that papers accepted by AAOS would demonstrate higher conversion to publication compared with OTA and AAHKS but expected abstract publication rates from OTA and AAHKS to be similar.
METHODS: All clinical and preclinical abstracts from the trauma and total joint arthroplasty subspecialties presented at the AAOS, OTA, and AAHKS annual meetings in 2015 were evaluated. Data collected included the current status of the publication, journal and publication date, time to publication, and country of origin (United States or international).
RESULTS: There were 516 (N = 213, AAOS; N = 303, OTA) trauma and 711 (N = 470, AAOS; N = 241 AAHKS) arthroplasty poster and podium presentations. When comparing publication rates in trauma, no significant difference was observed in overall publication rates between AAOS and OTA at 57.2% (N = 122 published) and 60.4% (N = 183 published), respectively (P = 0.54). In addition, no significant difference was observed in overall publication rates in arthroplasty between AAOS and AAHKS, with publication rates of 65.3% (N = 307 published) and 59.8% (N = 144 published), respectively (P = 0.17). Of abstracts that were published, AAHKS arthroplasty abstracts were more likely to be published in The Journal of Arthroplasty (JOA, 69.4%) compared with OTA trauma abstracts published in the Journal of Orthopaedic Trauma (JOT, 33.3%), P < 0.001.
CONCLUSION: The overall publication rates, along with publication rates to premiere subspecialty journals, is indicative of forefront research being presented at the three annual meetings. Given the comparable research quality of OTA and AAHKS abstracts, the AAOS meeting appears to provide the highest yield for surgeons with more generalized practices or practices spanning multiple subspecialties.

Like many other fields in medicine, orthopaedic surgery is continuously evolving with new ideas and research. Presentations at national orthopaedic conferences provide an important avenue through which to highlight and disseminate new and innovative research. For the researchers who present at these conferences, the eventual goal is to publish their findings in a quality, well-read, peer-reviewed journal. Many barriers exist to publication, and the number of abstracts never even submitted to journals for publication may be as high as one-third.[1] Especially for international researchers or private practice physicians, limited travel time and decreasing funding force many to select the optimal conference to attend or to submit research.

An important contributing factor to the practice of evidence-based medicine involves the publication of research in peer-reviewed journals. Although national conferences have their own peer review process that approves abstracts for presentation, projects presented at conferences may not be fully mature or may not pass the peer review process for a journal, and therefore never reach publication.[123456] The reported publication rates of different studies at various orthopaedic conferences have been shown to range from 28% to 67%.[2,789101112131415161718]

Studies have demonstrated an increasing trend in publication conversion rates for abstracts presented at the annual conference held by the American Academy of Orthopaedic Surgeons (AAOS) over the past 20 years.[1,3,5,6,8,19202122] However, several factors may contribute to the increase in the number of studies published within the field of orthopaedics, such as an increase in the number of journals and online, open-access options.[23] Few authors have reported impact factors (IFs) from abstracts published after presentation at orthopaedic meetings,[2,5,6,24,25] and the other surrogates for quality such as CiteScore (CS) and Scopus Source Normalized Impact per Paper (SNIP), which attempt to quantify and rank the quality of journals by analyzing citations normalized by the number of publications,[23,26272829] were not used.

The purpose of this study was to focus on abstracts in trauma and joint reconstruction. By evaluating studies presented at the AAOS, Orthopaedic Trauma Association (OTA), and American Association of Hip and Knee Surgeons (AAHKS) meetings, we sought to answer the following questions: What are the publication acceptance rates for abstracts presented at the AAOS, OTA, and AAHKS meetings? What are the differences in journal metrics for the published abstracts from each conference? What is the average time to publication (TTP)? What potential factors predict conversion to publication?

Methods

A review of all of the adult clinical and preclinical research abstracts accepted for either podium or poster presentation at the AAOS, OTA, and AAHKS conferences during 2015 was performed. Only research pertaining to orthopaedic trauma or arthroplasty was included in this study. Research abstracts that were filed under another subspecialty or under a forum other than trauma or arthroplasty were also included if they could also be categorized as trauma or arthroplasty. Eleven of 15 AAHKS poster abstracts filed under nonarthroplasty were excluded. Pediatric trauma was excluded, and video theater and scientific exhibits presented at AAOS were also excluded from the study. ScienceDirect, Ovid, Pubmed, and Google Scholar were queried to determine whether the research had been accepted for publication. The papers and posters were evaluated for the status of publication, publication date, journal in which the study was published, TTP, and country of origin. Studies conducted at institutions outside of the United States (US) were classified as international. In instances where authors collaborated between US and abroad institutions, the country of the senior author was used for analysis purposes. Time to publication was defined as the time between the first date of the conference and the date of publication in print of the manuscript. If this information was not available, then the acceptance date (if available) or online publication date was used. In instances of only the month and year of publication available, we used the first of the month. Initially, we searched for the published work using the complete conference abstract titles. If we could not find a paper with the same title, we pared down the title and included author names in the search criteria. If these methods did not yield results, a final effort was made searching institutional author profiles and bibliographies to review their published work for any matching criteria. When titles or authors differed between conference abstracts and journal publications, abstracts were compared to determine their similarities. Abstract titles, author groupings on abstracts, and digital object identifier for published abstracts were queried for matches between conferences using the Vlookup command with exact and approximate match options on Microsoft Excel (Redmond, WA).

Various journal metrics were used as a relative measure of the quality of the journal in which abstracts were published. The IF, recorded from the Clarivate Analytics journal database (Philadelphia, PA), the CS, the SNIP, and whether a journal was categorized as an Open Access Journal were recorded from the SCOPUS database (Elsevier, Atlanta, GA).

Time to publication, overall (any journal) publication rate, journal name, journal metrics, publication rates in the Journal of Orthopaedic Trauma (JOT), and The Journal of Arthroplasty (JOA) were compared. Descriptive statistics were performed using means and standard deviations for normally distributed data, and median and interquartile range (IQR) were used for nonparametric data. Categorical variables were compared using contingency tables. Data analysis was conducted using t-tests, Mann-Whitney U tests, chi-Squared tests, and Fisher exact tests where appropriate. A multiple logistic regression model was fit to identify any factors that may be predictive of publication. Associated odds ratios (ORs) and confidence intervals (CIs) were calculated for each factor. A Hosmer-Lemeshow test was performed on the regression model to asses for goodness of fit of the model. All analysis was conducted with an overall significance level of α = 0.05. The threshold for significance was corrected for multiple comparisons using the Bonferroni correction, αc = 0.0024.

Results

There were a total of 516 (N = 213, AAOS; N = 303, OTA) trauma and 711 (N = 470, AAOS; N = 241 AAHKS) arthroplasty poster and podium presentations (Table 1). Regarding international abstracts, there were a total of 146 (N = 58, AAOS; N = 88, OTA) trauma and 141 (N = 132, AAOS; N = 9, AAHKS) arthroplasty presented at these conferences.

Table 1

Summary of Conference Presentations and Publications With Journal Metrics Overall, 2015

Publication Rates

Overall, 60.4% of all abstracts accepted by OTA (N = 183) and 59.8% of abstracts accepted by AAHKS (N = 144) were published at median times of 306 (IQR: 75 to 469) days and 133 (IQR: 73 to 312) days after their respective conferences. Similarly, 62.8% of abstracts presented at AAOS (N = 429) were published at a median time of 148 (IQR: −34 to 446) days after the conference. Publication rates were similar between all conferences, P = 0.599. Fourteen published abstracts presented at OTA were also presented at AAOS. Three published abstracts presented at AAHKS were also presented at AAOS. Time to publication for abstracts presented at AAOS and AAHKS was similar, P = 0.76, and was quicker than that of abstracts presented at OTA (P = 0.002).

Twenty-nine percent, 18%, and 16% of the published AAOS, OTA, and AAHKS abstracts were published before their respective conferences. Of studies that were published, they mostly were published within a year of being presented with 70.2% AAOS, 57.4% OTA, and 78.5% AAHKS manuscripts being published within a year of their conference (Figure 1), and more than 90% from each meeting published by 2 years. The publication rates for trauma research presented at the AAOS and OTA meetings were similar at 57.2% and 60.4%, respectively (P = 0.54); likewise, the publication rates for arthroplasty research were similar at 65.3% and 59.8% for AAOS and AAHKS, respectively (P = 0.17).

Figure 1

Graph showing a survival curve illustrating the time taken for publication after conference date for all accepted conference manuscripts. The dotted lines represent the median time for publication for each respective conference.

For international abstracts, 57.9% (110/190) of AAOS abstracts and 62.5% (55/88) of OTA abstracts were published in a median time of 178 (IQR: −39 to 515) days and 315 (IQR: 29 to 441) days after their respective conferences. Six of 9 (66.7%) international abstracts from AAHKS were published in a median time of 71 (IQR: 98 to 157) days. Overall publication rates (P = 0.47) and TTP for international abstracts presented at AAOS and AAHKS were similar (P = 0.19), and also similar for those presented at AAOS and OTA (P = 0.47). International trauma abstracts presented at OTA (N = 55/88 published, 62.5%) were not more likely to be published than those presented at AAOS (N = 24/58 published, 41.4%), P = 0.01. International arthroplasty publication rates between AAHKS (N = 6/9 published, 66.7%) and AAOS (N = 86/132 published, 65.2%) were not different, P = 0.74. The percentages of the total accepted manuscripts as a function of publication time for international and US abstracts are presented in Figures 2 and 3, respectively.

Figure 2

Graph showing a survival curve illustrating the time taken for publication after conference date for accepted international manuscripts. The dotted lines represent the median time for publication for each respective conference.

Figure 3

Graph showing a survival curve illustrating the time taken for publication after conference date for accepted manuscripts for studies performed in the United States. The dotted lines represent the median time for publication for each respective conference.

Journal Metrics

Journal metrics and publication data for overall international presenters and US presenters are provided in Tables 1–3. Published abstracts presented at OTA had a significantly lower median IF compared with those published in AAOS (P < 0.001) or AAHKS (P < 0.001). The IF of publications between AAHKS and AAOS were similar (P = 0.003). AAOS abstracts had a significantly lower CS metric score compared with AAHKS, P < 0.001, but a significantly higher CS score compared with OTA, P < 0.001. Source Normalized Impact per Paper indicator scores were similar between abstracts published from the three conferences (P > 0.05). A total of 7.8% of published AAOS abstracts, 6.0% of OTA abstracts, and 2.8% AAHKS abstracts were published in open access journals.

Table 3

Summary of Conference Presentations and Publications With Journal Metrics for Studies Performed in the United States, 2015

Subspecialty Journal Publication

Of abstracts that were published, AAHKS arthroplasty abstracts were more likely to be published in The Journal of Arthroplasty, 69.4% (100/144), compared with OTA trauma abstracts published in the Journal of Orthopaedic Trauma, 33.3% (61/183), P < 0.001. Forty-four percent (135/306) of the published AAOS arthroplasty abstracts were published in JOA and 27.0% (33/122) of the published AAOS trauma abstracts were published in JOT. AAOS arthroplasty abstracts were less likely to be published in JOA compared with AAHKS (P < 0.001), whereas AAOS trauma abstracts had similar publication rates in JOT compared with OTA abstracts (P = 0.30). Trauma and arthroplasty abstract publication rates in The Journal of Bone and Joint Surgery (JBJS), Clinical Orthopaedics and Related Research (CORR), and Bone and Joint Journal (BJJ) were also compared. The proportions of studies published in these journals did not differ significantly between conferences (P > 0.0024 [all], Table 4). Publication differences in the Journal of the American Academy of Orthopaedic Surgeons were not assessed because the counts were too low for meaningful comparisons (less than 5 per conference).

Table 4

Summary of Abstracts Published in Leading Orthopaedic and Subspecialty Journals

Predictive Factors of Publication

Factors, associated ORs, and CIs when considering all abstracts in a multiple logistic model are reported in Table 5. Acceptance as a podium presentation was identified as a notable predictor of abstract conversion to publication (OR, 1.50; CI, 1.18 to 1.91). Accepted abstracts in arthroplasty trended toward having a higher likelihood of publication in comparison to trauma abstracts, (OR, 1.48; CI, 1.06 to 2.07). Neither being from a US institution nor an international institution predicted future publication (OR, 1.16, CI, 0.88 to 1.54). Acceptance by AAOS (OR, 1.17, CI, 0.84 to 1.63), OTA (OR, 1.40, CI, 0.86 to 2.28), or AAHKS was not a significant predictor of conversion to publication. The results of the Hosmer-Lemeshow test did not demonstrate a significant difference between observed and expected publication rates (P = 0.78).

Table 5

Analysis of Potential Predictive Factors of Publication After Multiple Logistic Regression for All Publications

Discussion

The purpose of this study was to compare AAOS, OTA, and AAHKS annual conference meetings by assessing their contributions to published research in the orthopaedic subspecialties of trauma and arthroplasty. We also aimed to demonstrate the overall quality of abstracts presented at these conferences using multiple different journal metrics. To our knowledge, this is the first study to broadly evaluate and compare publication rates, TTP, quality of journal of publication for trauma, and arthroplasty research presented at these major international orthopaedic and subspecialty meetings.

Our study indicates that publication rates between AAOS, OTA, and AAHKS are similar, with 60% to 63% of conference abstracts going on to publication in a peer-reviewed journal within two years. These data are consistent with other published results, supporting an increase in publication rates of conference abstracts[3,5,6,8,10,12,13,15,16,21] (Table 6). With the breadth of research submitted and presented at these meetings, review committees are seeking out higher quality research for presentation, which serves as a positive selection bias for publication. Likewise, with more options for publication, more high-quality research is likely to get published.

Table 6

Comparison to Previous Studies Investigating Conversion From Meeting Publication to Manuscript Publication

To assess the quality of research published from conference abstracts, various journal metrics and publication in open access journals were assessed. Based on study outcomes, acceptance by AAOS or AAHKS appears to be associated with publication in journals with higher IFs. The mean IFs of published manuscripts from all three conferences suggest that all three meetings present high-quality research because the journals of final publication have at least double the mean IF of other orthopaedic journals.[23] Although the validity of the IF and other journal metric scores (ie, CS and SNIP) have come under scrutiny in recent years, the higher values suggest the studies are reaching broader audiences.[23,26272829] Although there has been an increase in the number of open access journals, less than 10% of conference publications in our study were published in journals that identified as purely open access.

Our results show that podium presentations were more likely to result in full-text publication than poster presentations. This finding appears to validate the selection criteria of each of the three conferences in determining which studies are eventually going to be of the greatest impact.[9,11-13,15,16] This finding should be considered by meeting attendees because they choose how to spend their time at the annual meetings.

The TTP averaged less than two years regardless of the presentation format. This finding was in contrast to other studies with the times to publication of presentations for other orthopaedic subspecialty meetings.[2,6,8,12,13,15,16,20-22] Abstracts presented at AAOS and AAHKS were published in almost half the time as abstracts presented at OTA. It is unclear if submission deadlines or peer review process variations affected TTP.

Although overall abstract publication rates were comparable across the three meetings, there were some differences in where manuscripts were ultimately published. Abstracts from AAHKS were more likely to be published in JOA compared with abstracts from OTA being published in JOT. Arthroplasty abstracts from AAOS were also more likely to be published in JOA compared with trauma abstracts from AAOS being published in JOT. There was no difference in rates of publication for arthroplasty abstracts from AAHKS or AAOS or trauma abstracts from OTA or AAOS in JBJS, CORR, or BJJ. This discrepancy between arthroplasty and trauma abstracts in subspecialty journal publication rates could be due to a number of factors, including the peer review processes for the journals and conferences, choice of submission to the association-affiliated journal versus a nonaffiliated journal, intent to submit for publication, or intent to submit to general orthopaedic journals such as the Journal of the American Academy of Orthopaedic Surgeons or The Journal of Bone and Joint Surgery.

Publication rates for abstracts from outside of the United States were similar across the three conferences in our study, and did not differ markedly from publication rates for abstracts submitted from US institutions. Although publication bias is a concern for international authors, our study reported similar publication rates between international and US conference abstracts.[19,30] This supports the peer review process of the three conferences in our study in selecting quality research regardless of the country of origin.

The strengths of this study are in the reporting of publication rates and journal metrics over a range of conferences. The authors, however, acknowledge several limitations. First, we assumed that the authors presenting at these conferences actively sought publication of their research soon after presenting it. Second, if a conference abstract changed markedly from presentation to publication, we may not have identified it as a publication and therefore underestimated our publication rate. Third, we used IF, CS, and SNIP as surrogates for research quality without analyzing the quality of the study design. Finally, we only analyzed 1 year's worth of abstracts and two subspecialties, and annual presentation and publication fluctuations as well as specialty-related differences may occur.

Nevertheless, the comparative publication rates along with the early distribution of accepted manuscripts and high journal metrics of the associated journals are reflective of high-quality research that is consistently being presented at the three annual meetings. Given that publication rates, TTP, and journal quality are comparable between all three conferences, the AAOS meeting appears to offer the greatest breadth of high-quality research. Thus, although we recognize that there are multiple reasons for attending these meetings beyond learning of new research, surgeons with broader practices for whom this is the primary goal may benefit from attending the Academy because it will expose them to quality material from various subspecialties.

Table 2

Summary of Conference Presentations and Publications With Journal Metrics for International Presenters, 2015