Identification of abstract features presented at the combined otolaryngology spring meeting predicting publication in impactful peer-reviewed journals.

OBJECTIVES: Review abstracts presented at the Combined Otolaryngology Society Meeting (COSM) to determine subsequent publication and identify abstract features predictive of publication in high impact journals.
METHODS: A selection of abstracts accepted in the 2015 COSM scientific programs were retrospectively reviewed. MEDLINE searches via PubMed and Google Scholar were performed to determine publication rates. The Journal of Citation Reports was used to determine impact factors for published abstracts. Binomial regression analyses were used to identify factors related to publication in high impact journals.
RESULTS: 62.4% of reviewed abstracts (n = 623) were subsequently published, with a mean publication time of 14 ± 12 months. Abstract features predictive of publication were basic science type, other science type, prospective studies, multi-institutional involvement, and presentation at the American Laryngologic Association and American Otologic Society meetings. Based on Wald score, podium presentation was found to have the biggest effect on publication. Factors positively associated with publication in high impact (impact factor > 2.272) journals were increased author number and sample size.
CONCLUSIONS: Overall publication rate of abstracts selected for presentation at COSM in 2015 was on the higher end of previously reported otolaryngology meetings. Abstracts detailing basic science, other science, prospective and multi-institutional studies were more likely to lead to future publication. Additionally, increased number of authors and sample size lead to publication in higher impact journals. LEVEL OF EVIDENCE: N/A.

INTRODUCTION

Research presentation at scientific meetings is a valuable opportunity to disseminate scholarly work and presented material may affect the practice of clinicians in attendance. It must be noted, however, that content selected for scientific meetings is often presented before publication of the full‐text manuscript. Therefore, abstracts selected for conference presentation are anticipated to transition to final publication. Publication rates are an important quality metric for relevance and impact of selected research at a scientific meeting. This transition to publication, preferably in a peer reviewed journal, serves an important quality control mechanism for validating academic work. The peer review process improves published research quality, ensuring accuracy, appropriate and fair reporting of results, and conclusions that are reflective of presented data.

Selection of scientific meeting content is limited to review of concise abstracts summarizing unpublished work. Scientific planning committees must screen and select from thousands of submitted works based on abstract alone. Ideally, it would be useful to know of specific abstract features that predict subsequent publication, preferentially in high impact journals. This would be valuable information for selection committees to use as a frame of reference to ensure selected content is representative of subsequent high‐impact published work.

Reported publication rates from other major national academic Otolaryngology‐Head and Neck Surgery meetings is variable, ranging from 32% to 69%., , , , Combined Otolaryngology Spring Meeting (COSM) publication rates have not yet been reported. Previous reports have focused largely on publication rate and time to publication, with minimal attempts to quantify impact of subsequent published research. Additionally, attempts in otolaryngology to identify abstract features that predict publication is limited. A single identified study from 2006 attempted this and found that oral presentations with inferential statistics in the abstract and an increased number of authors were the most significant predictive factors for publication. Interestingly, level of evidence and direction of study inquiry were not reflective of publication likelihood.

We therefore sought to determine the rate and time to publication for content presented at a major Otolaryngology—Head and Neck Surgery conference: COSM 2015. More importantly, we aimed to review the individual abstracts selected for presentation to identify abstract characteristics that might predict subsequent high impact journal publication. Ultimately, the results of this study will further guide conference submission committees on key features to consider when selecting content for presentation, and serve as a reference for those submitting work about pertinent information to include in their crafted abstracts.

METHODS

Institutional Review Board approval was not required. Abstracts accepted and published in the 2015 COSM scientific programs for the following participating member societies were retrospectively reviewed: American Rhinologic Society (ARS), American Laryngological Association (ALA), American Head and Neck Society (AHNS), American Broncho‐Esophageal Association (ABEA), American Otological Society (AOS). Due to time constraints, only a portion of Triological Society (TRIO) abstracts were reviewed, randomly selected to provide a representative sampling. Selection of the year 2015 ensured an adequate interval to achieve publication while ensuring that a contemporary review of published material was conducted.

Abstract review was performed by four reviewers (FSG, CDD, VNY, DER). Initially, to standardize the review process, a selection of 10 abstracts was chosen and reviewed by each reviewer. Reviewers then compared their stratifications and discussed any differences in results and why these might have arisen. This established internal consistency for the review process. Following this, reviewers proceeded with independent abstract review. If uncertainties were encountered, they were discussed amongst reviewers and a consensus decision was finalized. Pre‐determined characteristics of each abstract were categorized (Microsoft Excel for Mac, Version 15.0, ©2017 Microsoft). Descriptions of each categorization is provided in Table 1. Studies that were retrospective analyses of a prospectively maintained data set were classified as prospective. Within the science type category, an abstract was classified as “other” if not basic or clinical science (ie, educational, translational, quality improvement, etc.). Studies were categorized based on whether they included any advanced statistical analysis, then further stratified into whether they included any statistically significant results. Advanced statistical analysis was considered as any analyses beyond that of descriptive methods (ie, mean, median, mode, percentages). A statistically positive result was considered P < .05. If the abstract qualified for more than one category, it was included multiple times (ie, a prospective, blinded, randomized control trial would be classified under three separate categories of prospective, blinding, and RCT). An article was considered a case report if fewer than three clinical subjects were provided. A study was considered completed if data collection was no longer ongoing at the time of abstract submission

TABLE 1

Overview of abstract review categories

Category	Stratification
Society	ABEA, ALA, TRIO, AOS, ARS, AHNS
Author number	Nominal scale
Presentation type	Podium, poster
Direction of inquiry	Retrospective, prospective
Science type	Basic, clinical, other
Sample size	Nominal scale
Survey	Yes, no
Systematic review	Yes, no
Meta‐analysis	Yes, no
Case report	Yes, no
Multi‐center	Yes, no
Randomized control trial	Yes, no
Blinding	Yes, no
Comparative group	Yes, no
Advanced statistical analysis	Yes, no
Statistically significant outcome	Yes, no
Study completed	Yes, no
Journal access	Open, closed

Abbreviations: ABEA, American Broncho‐Esophageal Association; ALA, American Laryngologic Association; AOS, American Otological Society; ARS, American Rhinological Society; AHNS, American Head and Neck Society; TRIO, Triological Society.

Review for publication in peer‐reviewed journals was performed between July 2019 and April 2020. Full abstract title was searched in PubMed and Google Scholar. If unsuccessful, additional search was performed using first author and keywords from the abstract title, then last author and key abstract title words, followed by screening through publications by the first and last authors associated with the abstract. A scientific meeting abstract was considered published if the abstract that appeared in a peer‐reviewed journal correlated with the subject matter, methods and the majority of authors listed in the scientific program.

For abstracts where no publications were identified, a second screen in a similar fashion was performed by a different reviewer (FSG or CDD) in April 2020. Additional permutations of abstract title key words and listed authors was performed to increase the yield of identified publications. This ensured all easily accessible peer reviewed publications up to April 2020 were included for statistical analysis.

For matched published work, the peer‐reviewed journal name, electronic publication dates, and journal type (open vs closed access) were noted. Time to first publication (electronic or print) was calculated in months from the presentation date at COSM 2015 (April 2015). Publications prior to the scientific meeting were recorded as 0 months. Journal impact factors (IF) for the year of initial publication were identified from the Journal Citations Report (JCR). If no JCR impact factor was listed, then the impact factor was sourced from a different reputable source.

Initial analysis of the database identified basic descriptive statistics for the overall publication activity associated with the 2015 COSM meeting. Binomial logistic regression of the database was then performed to evaluate the influence (odds ratios ±95% confidence interval) of the identified categories of abstract features on subsequent publication status. Binomial logistic regression analysis was utilized due to the dichotomous nature of the dependent variables in the present investigation (ie, published vs not published, high‐ vs low‐IF), and because the abstract feature categories involved independent variables that were either continuous or categorical in nature. Statistical interrogation of the database occurred in two distinct phases. In the first phase, the binomial logistic regression analysis utilized the complete array of abstract feature categories as the covariate analysis set. From this initial run, any categories with overarching statistically significant effects on publication status were noted. To probe for additional, granular abstract feature categories that potentially influenced publication status, the categories with large overall effect sizes were removed from the covariate set for the second phase of the analysis. An additional binomial logistic regression was then performed. This same stepwise sequence of binomial logistic regression analyses was also utilized to interrogate the database for abstract features associated with manuscripts that were subsequently published in higher‐ vs lower‐IF journals. The median impact factor was used as a cut point for classifying IFs as either higher or lower. The median (rather than mean) was specifically selected due to the presence of a skew in the frequency distribution, and to create two groups of roughly similar size for use in the binomial analyses. All statistical analyses of the abstract database were performed using IBM SPSS Statistics for Windows, version 26 (IBM Corp., Armonk, New York). Statistical significance for all analyses was established using a two‐sided α = .05.

RESULTS

A total of 623 abstracts were reviewed; 178 (28.6%) podium presentations and 445 (71.4%) posters. Publication outcomes, stratified by society, are summarized in Table 2. Subsequent publication rate from reviewed abstracts was 62.4%, with the majority in closed access journals (93.2%). Of all 389 publications, 61.7% were derived from posters, and 38.3% from podiums. Nonetheless, publication rate was higher for podium presentations (83.7%) than posters (55.2%). Mean time to publication was 14 ± 12 (range 0‐59) months. Mean journal IF for published work was 2.28 ± 1.40 (range 0.18‐17.14; lower quartile 1.51, upper quartile 2.47). Journals in which abstracts were most frequently published (and associated 2015/2016 IFs for reference) are summarized in Table 3.

TABLE 2

Comparison of 2015 COSM publication rates stratified by individual society meetings

Society	No. abstracts in scientific program	No. of abstracts reviewed	Proportion of program abstracts reviewed	No. of abstracts published	Publication rate	Mean time to publication (months)	Mean impact factor
ABEA	93	92	98.9%	52	56.5%	16	1.489
ALA	95	95	100%	73	76.8%	11	2.115
TRIO	173	42	24.3%	25	59.5%	14	1.950
AOS	59	59	100%	43	72.9%	13	1.921
ARS	153	153	100%	93	60.8%	13	2.334
AHNS	182	182	100%	103	56.6%	15	2.977
Total	—	623	—	389	62.4%	14	2.282

Abbreviations: ABEA, American Broncho‐Esophageal Association; ARS, American Rhinological Society; ALA, American Laryngologic Association; AOS, American Otological Society; AHNS, American Head and Neck Society; TRIO, Triological Society.

TABLE 3

Top 5 journals for published 2015 COSM abstracts

Journal	Proportion of published abstracts	Affiliated society	Impact factor
			2015	2016
Laryngoscope	20.1%	TRIO, ALA	2.272	2.471
International Forum of Allergy and Rhinology	12.6%	ARS	2.350	2.135
Annals of Otology Rhinology and Laryngology	9.5%	ABEA	1.292	1.384
JAMA Otolaryngology‐Head and Neck Surgery	7.2%	AHNS	2.705	2.951
Otology and Neurotology	6.9%	AOS	1.935	2.024

Abbreviations: ABEA, American Broncho‐Esophageal Association; ARS, American Rhinological Society; ALA, American Laryngologic Association; AOS, American Otological Society; AHNS, American Head and Neck Society; TRIO, Triological Society.

Nearly all abstracts were for completed studies (95.5%) at the time of submission. The vast majority had multiple contributing authors (97.4%) listed, with mean authorship of 4.65, range 1‐17. Studies were deemed clinical science (79.8%), basic science (16.2%), or other (4.0%). The direction of study was classified as either retrospective (55.4%) or prospective (44.5%). The study sample size was clearly stated in most abstracts (89.1%), with a median 46.5 subjects (range of 1‐4 182 904; lower quartile 13.75, upper quartile 160.5). Study abstracts included case reports (7.2%), surveys (4.0%), randomized control trials (1.8%), systematic reviews (2.6%), and meta‐analyses (0.3%). Multi‐institutional data was reported in 15.6% of the reviewed abstracts. A comparative group was included in 49.4%, and only a minority specifically mentioned blinding of subjects/investigators (2.9%). Use of advanced statistical analysis was included in 49.6% of abstracts, with statistically significant findings (P < .05) reported in 42.2%.

Two abstracts (0.3%) coded as meta‐analysis were excluded from statistical analysis given their disproportionately low incidence in comparison to other evaluated abstract variables. Binomial logistical regression analysis of publication success for the included 621 abstracts was performed (Table S1). The model accounted for 70.0% of the variance in publication success and represented a statistically significant improvement over the randomly generated baseline model (χ 2 = 123.562, df = 21, P < .001). Statistically significant category effects on publication success were observed for society (Wald χ 2 = 15.917, df = 5, P = .007) and science type (Wald χ 2 = 13.622, df = 2, P = .001). Within the society category, positive, statistically significant associations with publication success were found for presentations at the ALA (OR = 2.86, P = .002) and AOS (OR = 2.25, P = .035) meetings. Within the larger category of science type, both basic science (OR = 2.65, P = .002) and other science (OR = 4.82, P = .018) were positively and significantly associated with publication success. Positive statistical significance was additionally found for podium presentations (OR = 3.38, P < .001) and multi‐institutional studies (OR 1.99, P = .017).

A second binomial logistical regression analysis of publication success (Table 4) was performed excluding categories of society and science type from the set of covariates given their large, overall category effects on the analysis. This binomial logistic regression model accounted for 66.0% of the variance in publication success. Statistically significant, positive associations were found for prospective studies (OR = 1.57, P = .020) and podium presentations (OR = 3.64, P < .001). Statistically significant, but negative associations were found for studies incomplete at the time of abstract submission (OR = 0.36, P = .031).

TABLE 4

Binomial logistical regression analysis of factors related to publication status (excluding society and science type)

Factor	Wald χ 2	p value	Odds ratio (95% CI)
Author number	2.112	0.146	1.07 (0.98, 1.17)
Podiuma	30.060	<0.001	3.64 (2.29, 5.78)
Prospectiveb	5.411	0.020	1.57 (1.07, 2.29)
Sample size	2.093	0.148	1.00 (1.00, 1.00)
Surveyc	0.019	0.892	0.94 (0.37, 2.36)
Systematic reviewc	3.179	0.075	3.44 (0.88, 13.38)
Case reportc	1.871	0.171	0.61 (0.30, 1.24)
Multi‐centerc	2.459	0.117	1.55 (0.90, 2.67)
RCTc	0.131	0.718	0.75 (0.16, 3.60)
Blindingc	0.002	0.961	1.03 (0.30, 3.54)
Comparative groupc	0.473	0.491	0.87 (0.57, 1.31)
Advanced statistical analysisc	0.330	0.566	1.22 (0.62, 2.38)
Statistically significant outcomec	1.037	0.308	1.41 (0.73, 2.73)
Study completedc	4.636	0.031	0.36 (0.14, 0.91)

Note: Reference categories: a Poster presentations, b Retrospective studies, c Binary, nominal variables that were coded as “Yes” or “No” on the basis of the presence/absence of the category feature in the abstracts evaluated. For all categories, with the exception of “Study completed,” the “No” condition was used as the reference category. *Model Summary: χ2 = 91.063 (df = 14), P < .001.

Abbreviations: CI, confidence interval; RCT, randomized clinical trial.

Table 5 summarizes binomial logistic regression analysis results correlating factors predictive of publication in a high IF journal. Journal IFs were classified as a bivariate outcome (high vs low IF) based on a median journal IF of 2.272 for published abstracts. A total of 380 abstracts were included in the analysis (excluding 234 unpublished abstracts and 9 published abstracts with unavailable journal IF). The model accounted for 77.4% of the variance in IF status and represented a statistically significant improvement over the randomly generated baseline model (χ 2 = 137.042, df = 23, P < .001). Statistically significant positive associations were found for author number (OR = 1.18, P = .026) and sample size (OR = 1.001, P = .036). Statistically significant negative associations were found for presentation at the ABEA (OR = 0.062, P < .001) and AOS (OR = 0.013, P < .001) meetings, and publication in an open access journal (OR = 0.240, P = .014). Repeat analysis excluding society presentation and science type (as performed previously for publication success) yielded similar results (data not shown); no additional positive associations were identified.

TABLE 5

Binomial logistical regression analysis of factors related to publication in high impact journal (IF > 2.272)

Factor	Wald χ 2	P value	Odds ratio (95% CI)
Society (ref: AHNS)	57.323	<.001
ABEA	32.357	<.001	0.06 (0.02, 0.16)
ALA	0.019	.891	0.94 (0.41, 2.16)
TRIO	1.891	.169	0.47 (0.16, 1.38)
AOS	27.057	<.001	0.01 (0.00, 0.07)
ARS	3.001	.083	0.53 (0.26, 1.09)
Podiuma	2.070	.150	1.57 (0.85, 2.90)
Prospectiveb	0.006	.938	0.98 (0.56, 1.72)
Science type	0.156	.925
Basic sciencec	0.001	.978	0.99 (0.46, 2.15)
Other sciencec	0.148	.701	1.25 (0.40, 2.86)
Surveyd	0.284	.594	0.71 (0.20, 2.55)
Systematic reviewd	1.987	.159	0.38 (0.10, 1.46)
Case reportd	0.367	.545	0.63 (0.15, 2.77)
Multi‐centerd	0.563	.453	1.32 (0.64, 2.74)
RCTd	<0.001	.990	0.99 (0.14, 7.13)
Blindingd	0.033	.855	0.86 (0.17, 4.40)
Comparative groupd	0.079	.778	1.09 (0.60, 1.98)
Advanced statistical analysisd	0.074	.785	0.88 (0.34, 2.26)
Statistically significant outcomed	<0.001	.983	0.99 (0.41, 2.42)
Study completedd	1.399	.237	3.29 (0.46, 23.71)
Open accessd	6.072	.014	0.24 (0.08, 0.75)
Author number	4.934	.026	1.18 (1.02, 1.35)
Sample size	4.382	.036	1.001 (1.000, 1.002)
Time to publication	0.635	.425	(0.99 (0.97, 1.01)

Note: Reference categories: a Poster presentations, b Retrospective studies, c Clinical science, d Binary, nominal variables that were coded as “Yes” or “No” on the basis of the presence/absence of the category feature in the abstracts evaluated. For all categories, with the exception of “Study completed” and “Open access,” the “No” condition was used as the reference category. *Model Summary: χ 2 = 137.042 (df = 23), P < .001.

Abbreviations: CI, confidence interval; IF, impact factor; RCT, randomized clinical trial.

DISCUSSION

This study provides a contemporary assessment of factors of abstracts selected for presentation at a major otolaryngology scientific meeting that predict subsequent publication. For reference, 62.4% of reviewed 2015 COSM abstracts proceeded to publication. The overall publication rate compares favorably in comparison to other reported Otolaryngology‐Head and Neck Surgery national meetings (32%‐50% for AAO, 51% for CSO‐HNS 51%, and 69% for UK ORS)., , , Mean time to publication following presentation was 14 ± 12 months, more expeditious than previous reports.,

Based on Wald score, podium presentation was found to have the biggest effect on publication success (83.7%, OR 3.64). This is also a consistent finding among most reports in other specialties assessing factors predicative of full text manuscript publication., , , , , This is encouraging, suggesting that selection committees have well‐established criteria and a rigorous process for identifying higher‐quality research for oral presentation. Oral presentation aside, predictive factors of future publication found for this Otolaryngology based national meeting included prospective and multi‐institutional studies, as well as studies in the basic and other science categories. Our study suggests these study abstract features should be valued when selecting for podium presentations. In contrast, abstracts incomplete at the time of submission were less likely to be published. This may be because the research was still ongoing, or, alternatively, due to difficulty with study completion because of lost interest or inadequate time for concluding research and manuscript preparation., Thus, selection of abstracts with incomplete or ongoing collection of study results should be made with caution, especially so as to avoid mis‐representation of conclusions from incomplete, unpublished data.

The literature reports variable predictors of publication success beyond selection of oral presentation in other specialties. Commonly identified abstract features portending subsequent publication include basic science research, randomized trail design, and research performed in the academic setting., , , , A large Cochrane meta‐analysis of over 300 000 abstracts concluded that “positive” results (defined as any “significant” result), oral presentations, acceptance for meeting presentation, randomized trial design, basic research and academic setting research were all factors favorably associated with full publication. There was no attempt made, however, at correlating these features to publication impact factors.

Herein lies a unique aspect of our study. We identified correlations not only with publication rate, but also journal IF. Publication in high impact journals is desirable, as this may portend subsequent citations and reflect well on one's academic performance. Our median journal IF for published work was 2.27. Author number and sample size correlated with publication in higher impact journals. We conjecture that increased number of contributing authors provides improved study perspective and critical appraisal, refining the work's overall quality. Additionally, larger sample sizes inspire increased confidence in a study's findings and conclusions. In contrast, a negative association was found for publication in open access journals, meaning abstracts selected for publication in such journals were of lower IF. This suggests higher quality research is preferentially submitted and accepted for publication in closed access journals, where the IFs are generally higher.

Previous studies have shown higher publication rates for abstracts including statistical analysis,, intuitively adding validity and power to a study's methodology and data analysis. We stratified abstracts based on incorporation of advanced statistical analysis, defined as anything beyond basic descriptive statistics. Approximately half the abstracts reviewed included advanced statistical analysis (49.6%). Our regression analysis, however, did not find this to be a significant predictor. Perhaps statistical details and reporting of statistically positive results are omitted from many abstracts for the sake of brevity, therefore showing poor correlation with publication success.

Interestingly, when poster vs podium was attempted to be removed from each iteration of the regression analysis, the statistical model unfortunately lost significant predictive ability, and was ultimately unable to adequately explain the variance in publication outcomes. This suggests that there are factors beyond the granular variables assessed in our methodology contributing to publication success. Features such as novel concepts or study questions, value of contribution added to already available literature, and contemporary/current topics are more subjective in nature and difficult to quantify categorically. These features are often assessed qualitatively, both during abstract review and during the final peer review process, and are likely important for achieving publication.

Several study limitations deserve recognition. Ideally, we would have included all society programs and abstracts for review over many COSM meetings for comparison. However, time limitations affected the feasibility of performing such a large volume for accurate and comprehensive review. Additionally, we did not include level of evidence as a category, given this is often not explicitly stated in abstracts, and can be challenging to determine consistently. Despite this, most of the variables that determine level of evidence are inherently related to the other categories we evaluated. For science type, our methodology could have been improved by further stratifying other science type into additional sub‐categories (ie, quality improvement, education, translational, etc.). Depending on the quality and information contained in the abstract itself, some categories were unable to be determined reliably for every individual abstract. Finally, acceptance for presentation by a society at COSM often mandates submission of presented work to a specific journal affiliated with that society. Therefore, an inherent association of society and impact factor is inevitable. Areas of future investigation include trying to incorporate number of individual references/citations of a published work as an outcome measure. The reasoning here is that specific publications more frequently cited (regardless of journal IF) are likely of higher quality and clinical relevance.

CONCLUSIONS

Publication rate of abstracts presented at the 2015 COSM was higher compared to other major academic Otolaryngology‐Head and Neck Surgery meetings. Specifically, abstracts in the basic and other science, prospective, multi‐institutional and podium presentation categories were most likely to lead to future publications. Conversely, abstracts that were incomplete at the time of submission were less likely to lead to future publications. Author number and sample size were positive predictors of publication in a high‐impact journal, whereas publication in an open access journal was a negative predictor. Our study findings serve as a basis for selection committees when evaluating future conference submissions to guide identification of research likely to lead to publication.

CONFLICT OF INTEREST

Clark A. Rosen discloses the following disclosures and financial relationships: Olympus America Inc: Consultant, Instrumentarium: Royalties, Freundenberg Medical: Consultant, Reflux Gourmet LCC: Shareholder. The other authors have no financial relationships or conflicts of interest to disclose.

ETHICS APPROVAL

Not applicable.

CONSENT FOR PUBLICATION

Not applicable.

Supplemental Table S1 Binomial logistical regression analysis of factors related to publication status

Click here for additional data file.