Publication rates for abstracts presented by Korean investigators at major radiology meetings.

OBJECTIVE: To determine the publication rate of abstracts presented by Korean investigators at national and international radiological meetings, and to identify predictive factors of publication.
MATERIALS AND METHODS: Abstracts presented at the annual meetings of the Korean Radiological Society (KRS), and abstracts presented by Korean investigators at the annual meetings of the Radiological Society of North America (RSNA) and European Congress of Radiology (ECR) from 2001 to 2002 were searched for subsequent publication, using PubMed and the Korean Medical Database. The following variables were evaluated. 1) The overall publication rate; 2) the publication rates according to the radiological subspecialty, presentation type (oral or poster), sample size (< or = 20, 21-50, or > 50), study design (prospective or retrospective), statistical analysis (present or absent), and study outcome (positive or negative); 3) the time to publication; 4) the journal where the study was published; 5) consistency between the abstract and the final publication.
RESULTS: Of 1,097 abstracts, 301 (27.4%) were subsequently published, at an average of 15.8 +/- 13.8 months after presentation in 48 journals. The publication rates for studies presented at the RSNA (35.4%) and ECR (50.5%) conferences were significantly higher than that for the KRS conference (23.6%, p < 0.05). Vascular/interventional radiology studies had the highest publication rate (33.1%), whereas musculoskeletal radiology studies had the lowest publication rate (17.1%). Other factors associated with subsequent publication were prospective design, use of statistical testing, and a positive study outcome.
CONCLUSION: The publication rate is significantly lower for the KRS (23.6%) meeting abstracts as compared to those of the RSNA (35.4%) and ECR (50.5%). Prospective design, use of statistical testing, and positive study outcome have a statistically significant effect on the publication rate.

Presentations at national and international meetings provide an important forum for the dissemination of current research findings to the scientific community. Subsequent publication in a scientific journal is the natural and final outcome of such presentations, where the investigation is subjected to exhaustive manuscript preparation, extensive analysis of results, and critical peer review process. The publication rate of presentations may be regarded as an indicator of the quality of the scientific research presented at the meeting (1-6) and of the country where the research was performed (7-9).

However, the publication rate of presentations have ranged from 11% to 78% depending on medical specialty, with an average of 45% (1); these findings suggest that more than half of all abstracts are not fully published in scientific journals after presentation at the conferences. To the best of our knowledge, only one study (10) has evaluated the publication rate of scientific presentations at the Korean national radiology meeting. Furthermore, the publication rates of abstracts presented by Korean investigators at international radiological meetings have not been systematically investigated.

The aims of the present study were to determine the publication rate of abstracts presented by Korean investigators at a national conference (annual meeting of the Korean Radiological Society, KRS) and at two major international radiology conferences (the annual meeting of the Radiological Society of North America, RSNA and the European Congress of Radiology, ECR), and to identify factors predictive of publication.

MATERIALS AND METHODS

Data Collection

All abstracts at the KRS conference and the abstracts presented by Korean investigators at the RSNA and ECR from 2001 to 2002 were identified, using a search of the conference program books containing the abstracts. In addition, a search of the official websites of the KRS (http://www.radiology.or.kr/event/annual.html), the RSNA (http://archive.rsna.org/index.cfm), and the ECR (http://www.myesr.org/cms/website.php?id=/en/congress/ecr_2007/about_the_congress/past_meetings.htm) was performed on the Internet. We only included original research studies. Abstracts that illustrated an imaging technique or reported case(s) of educational value or special interests were excluded. When more than one country was listed on the abstract, we considered that the abstract originated from Korea if more than 50% of authors were Korean nationals (with a typical Korean name). The abstracts that were withdrawn were not included in the study.

A computer-based search for each abstract was performed to determine whether it was published in a peer-reviewed journal. The publication status was assessed by using the PubMed database (http://www.ncbi.nlm.nih.gov/PubMed/) and the Korean medical database (http://kmbase.medric.or.kr/) from January 1, 2000 to June 30, 2007 (the date of completion of the search). Therefore, the follow-up periods ranged from 55 months (for the RSNA 2002 conference) to 76 months (for the ECR 2001 conference). The searches began with the first name initial(s) and the full last name of the first author. If no corresponding publication was found, this search was followed by a search for subsequent authors or keywords from the title of the abstract. A published manuscript was considered as a full publication of a corresponding abstract when it satisfied the following two criteria: 1) at least one author on the abstract was listed as an author of the full publication, and 2) at least one conclusion from the presented abstract was included in the conclusions of the final publication. Six investigators (an investigator for each of the six meetings) performed primary data curation. Another investigator performed validation of the data from the abstracts and differences were resolved by consensus. Institutional review board approval was not required for this retrospective analysis.

Data Analysis

The following variables were evaluated: 1) the overall publication rate (the ratio of the number of subsequent publications to the total number of abstracts) for each of the respective meetings; 2) publication rates according to the classification of presentations; 3) time to publication; 4) journals in which the study was published; 5) consistency between the abstract and the final publication.

Based on the outcome of our search, overall rates of publication were calculated for each of the respective meetings. These publication rates were compared with each other and with the rates reported for other radiological and other medical disciplines.

For evaluation of publication rates according to the classification of presentation, we collected the following information from each abstract: radiological subspecialty, presentation type (oral or poster), sample size (≤ 20, 21-50, or > 50), study design (prospective [including experimental studies] or retrospective [including no available information]), statistical analysis (present [authors stated the method of statistical analysis used or reported p values] or absent), and study outcome (positive [the studied variables produced beneficial or statistically significant results] or negative).

For purposes of analysis, the abstracts were subdivided into the following 10 radiology subspecialties based on generally accepted categorizations often found in the literature as well as on the grouping of abstracts in the final program of each respective meeting. These subspecialties included breast, cardiac, chest, gastrointestinal, genitourinary, musculoskeletal, neuroradiology/head and neck, pediatric, vascular/interventional, and others (including nuclear medicine, physics, basic science, and radiation oncology).

The time to publication was defined in months for the duration between the month of publication and the month of abstract presentation. When a journal was published every two months, we defined the time of publication as occurring halfway between the two months. If a publication had occurred in the same month as the meeting or any time before the abstract presentation, the time to publication was recorded as zero months.

The journal impact factor of Medline-indexed journals was retrieved from the Science Citation Report on the Thompson Scientific ISI Web of Knowledge server (http://scientific.thompson.com/webofknowledge/). This impact factor is calculated by dividing the number of citations in the current year to the number of articles published in the past two years. The mean impact factor for each journal between 2002 and 2006 was calculated and expressed as the impact factor value.

We also evaluated consistency between the abstract and the final publication, including differences in the title, the number of authors, the position of the first author in the abstract, study objective/hypothesis, study design, sample size, statistical analysis, study results, and conclusions. When a full publication was confirmed, two investigators independently compared the content of the abstracts of the presentations and the summaries of the published studies; discrepancies were resolved by discussion with another investigator.

Statistical analyses for overall publication rates and predictive factors for publication were determined using χ2 tests. A p value < 0.05 was considered as statistically significant. All statistical analyses were performed by using SPSS statistical software (version 10.0; SPSS, Chicago, IL).

RESULTS

Publication Rate

A total of 1,230 abstracts (732 oral and 498 poster presentations) presented at the six meetings were identified. One hundred thirty-three posters presented at the KRS meeting were excluded from the study (108 educational exhibits, 17 technical exhibits, and 8 case reports). Thus, the remaining 1,097 abstracts constituted the basis of this study. Of these abstracts, 301 were subsequently published as full-text articles by June 30 2007, as determined by the computerized search, giving an overall publication rate of 27% (Table 1).

Table 1

Overall Publication Rates for Each Meeting

Note.-Numbers in parentheses are publication rates (%). *Significantly lower than RSNA and ECR (p = 0.002 and p < 0.001, respectively), †Significantly lower than ECR (p = 0.029). KRS = the annual meeting of the Korean Radiological Society, RSNA = the annual meeting of the Radiological Society of North America, ECR = European Congress of Radiology

The publication rate for studies presented at the ECR conference (50.5%) was significantly higher than the rate for the RSNA conference (35.4%; p = 0.029) and KRS conference (23.6%; p < 0.001). In addition, there is a statistically significant difference between the publication rate for the RSNA and KRS conferences (p = 0.002). However, there was no significant difference in publication rates between abstracts for oral (28.6%) and poster presentations (25.2%; p = 0.272).

Predictive Factors for Publication

Table 2 shows publication rates according to subspecialty. "Vascular/interventional radiology" had the highest publication rate (33.1%), whereas "musculoskeletal radiology" had the lowest publication rate (17.1%). Studies in the field of "musculoskeletal radiology" were published significantly less often than the mean (p = 0.03).

Table 2

Publication Rates According to Radiology Subspecialty

Note.-Numbers in parentheses are percentages (publication rates). *Significantly lower than the mean (p = 0.03). KRS = the annual meeting of the Korean Radiological Society, RSNA = the annual meeting of the Radiological Society of North America, ECR = European Congress of Radiology

Publication rates according to study variables are shown in Table 3. Studies described in abstracts with a prospective design had higher publication rates than studies with a retrospective design (33.4% vs 25.1%, respectively, p = 0.007). There was also a significant difference in publication rates based on statistical analysis and study outcome (p < 0.0001 and p = 0.0001, respectively). However, the sample size did not significantly influence the publication rate (p = 0.136).

Table 3

Publication Rates According to Study Variables

Note.-Numbers in parentheses are publication rates (%).

For published articles, the time course between abstract presentation and journal publication for each meeting is shown in Table 4. Interestingly, 41 (3.7% of all presented abstracts and 14% of all published papers) were published before the date of abstract presentation at a scientific meeting. The overall mean time to publication was 15.8 months (standard deviation, 13.8 months), when studies published before the meeting were given a time to publication value of zero. Most (81.7%) of the studies presented in the abstracts were published within two years of presentation. Only 10% of the studies presented in the abstracts were published more than three years after presentation.

Table 4

Publication Rates Divided According to Time from Meeting*

Note.-Numbers in parentheses are publication rates (%). *Cumulative percentages of publications are provided for years following respective meetings. Last numbers shown (in > 4 years row) is overall number of publications and publication rate. †Thirty presentations were published before meeting at which they were given. All presentations were followed for at least 55 months; presentations from RSNA 2002 were followed for 55 months, and presentations from ECR 2001 were followed for 76 months. KRS = the annual meeting of the Korean Radiological Society, RSNA = the annual meeting of the Radiological Society of North America, ECR = European Congress of Radiology

Table 5 lists journals that published studies for each meeting and the impact factors. Two-hundred studies from the KRS meetings were published in 45 different journals. Seventy-six studies (38.0%) were published in Journal of the Korean Radiological Society (JKRS). In contrast, 101 studies from the RSNA and ECR meetings were published in a total of 27 journals including, in decreasing order of frequency, American Journal of Roentgenology (n = 23, 22.8%), Radiology (n = 21, 20.8%), JKRS (n = 15, 14.9%), and Korean Journal of Radiology (n = 9, 8.9%).

Table 5

Distribution of Publications in Journals from Each Meeting

Note.-Numbers in parentheses are percentages. *Mean impact factor from 2002 to 2006. †Each of these journals published fewer than three abstracts. KRS = the annual meeting of the Korean Radiological Society, RSNA = the annual meeting of the Radiological Society of North America, ECR = European Congress of Radiology, AJR = American Journal of Roentgenology, AJNR = American Journal of Neuroradiology, JCAT = Journal of Computer Assisted Tomography, Eur Radiol = European Radiology, JUM = Journal of Ultrasound in Medicine, KJR = Korean Journal of Radiology, J Kor Radiol Soc = Journal of the Korean Radiological Society, J Kor Soc Ultrasound Med = Journal of Korean Society of Ultrasound in Medicine

We sought to identify any inconsistencies between the abstracts and the subsequent full-text publications. Study information was consistent for the position of the first author, study objective/hypothesis, design, statistical analysis, results, and conclusions (Table 6). In contrast, investigators altered presentations by changing the title for 41% of the studies, adding or deleting authors for 79%, of the studies and changing the sample size for 42% of the studies.

Table 6

Consistency of Abstract Presentations with Final Publications

Note.-KRS = the annual meeting of the Korean Radiological Society, RSNA = the annual meeting of the Radiological Society of North America, ECR = European Congress of Radiology

DISCUSSION

The annual KRS meeting is a national radiology meeting that provides an important forum for the dissemination of current research findings in Korea. The RSNA and ECR meetings are two internationally prominent meetings in the field of radiology. Recently, many abstracts have been presented by Korean investigators at both scientific meetings, and the number of presentations from Korea has increased gradually.

Our survey of 1,097 abstracts from three radiology meetings revealed that 27% of abstracts presented at these meeting in 2001 and 2002 were subsequently published in peer-reviewed journals. In comparing the publication rates between the three meetings, the publication rates from two international meetings (RSNA and ECR) were significantly higher than that from the KRS meeting, perhaps because fewer but better quality studies were presented at international meetings. Moreover, the publication rate for studies presented at the ECR conference was significantly higher than that for the RSNA conference, although the exact reason for the difference is unclear.

A few studies that were performed in the field of radiology revealed the publication rate for presentations at international meetings to be between 33% and 47% (Table 7) (5-8, 11), although all studies included only oral presentations. In our series, the publication rate in Medline-indexed journals of abstracts presented by Korean investigators at the RSNA and ECR meetings was 33% (83/249). Although this rate is lower than the rates for the ECR 2000 and 2001 conferences (47% and 45%, respectively) (7, 8), it is comparable to those observed at other USA-based national and international radiological meetings (33-40%).

Table 7

Reported Publication Rates of Presentations at National or International Radiological Meetings

Note.-JFR = Journées Francaises de Radiologie, TCR = Turkish Congress of Radiology, KRS = annual meetings of the Korean Radiological Society, JKRS = Journal of the Korean Radiological Society, RSNA = annual meeting of the Radiological Society of North America, RANZCR = annual meeting of the Royal Australian and New Zealand College of Radiologists, ASNR = annual meeting of the American Society of Neuroradiology, ESGAR = annual meeting of the European Society of Gastrointestinal and Abdominal Radiology, ECR = European Congress of Radiology

Yun et al. (10) reported that 25% of studies described in abstracts that were presented at the 1992-1996 KRS meetings were subsequently published in JKRS. The publication rate in JKRS as determined in the present study from the KRS meeting was 9% (76/848). A possible explanation for this difference may include that many researchers published research in the Medline-indexed journals, or Korean researchers have less interest in publishing studies in JKRS. In contrast, the publication rate in Medline-indexed journals for studies described in abstracts presented at the 2001 and 2002 KRS meetings was 12% (102/848). This publication rate is comparable to rates reported for a meeting in France (12) and Turkey (13), but markedly lower than the rates rate for Australian and New Zealand meetings (4) (Table 7).

Failure to publish a study originally presented in an abstract is problematic for a variety of reasons. First, although some journals publish the abstracts of society meetings, in general the abstracts of meetings may be available only to meeting attendees. These abstracts are therefore of little use to the general medical and scientific community. Second, the abstracts presented at scientific meetings usually have not undergone rigorous peer review, and lack the necessary detail for readers to critically appraise a given study for its validity. Third, nonpublication of negative results may lead to publication bias (14, 15). In the field of radiology, it can result in an overestimation of the precision of a radiological technique. Last, failure to publish may be unethical and wasteful, leading to the potential replication of almost identical studies.

In this study, we documented that only a relatively small percentage of studies presented at scientific meetings were finalized in the form of a published paper. Although most investigators have identified "lack of time" as a reason for not publishing findings presented in abstracts, actual reasons are multifactorial. Other reasons for failure of publication include low priority, ongoing preparation of results, lack of funds or other resources, lack of faith in the quality of research, rejection of a submitted paper, problematic relationships with co-authors, negative results, and the existence of other published reports with identical results (1, 16-19).

In this study, we included six study variables (presentation type, radiological subspecialty, sample size, study design, statistical analysis, and study outcome) to identify factors predictive of publication. Our findings suggest that a prospectively designed study, use of statistical testing, and positive outcome were significant predictors of publication. These results are similar to those found in some previous analyses (1, 6, 15, 20-23).

In addition, studies in oral presentations are more likely to be published in peer-reviewed journals than studies in poster presentations, although this difference did not reach significance. The influence of an oral presentation on publication has been well documented in other specialties (1, 19, 20, 24-26). One might hypothesize that high-quality research abstracts would be selected for oral presentation by the program committee and authors of studies not selected for oral presentation are given the option of presenting a poster.

The topic of the presentation may be related to the publication rate, but in our study the publication rate according to the 10 different subspecialties did not markedly differ. Only the musculoskeletal radiology subspecialty had a markedly lower publication rate than the other subspecialties.

Although we did not specifically evaluate all variables that may influence the publication rate, a broad spectrum of predictor variables influencing publication among presentations at meetings has been identified. Investigators of previous studies have reported that country of origin (7-8), basic research (27), originality of a study (19), affiliation with a university (28), a randomized study (28), external financial support (22, 29), and international collaboration (9) appeared to influence the publication rate.

We found a substantial change in sample size and authorship from the time of initial presentation to final publication. The sample size of a study changed in 43% of the published studies originally presented as abstracts; this finding is similar to reported values of 39-46% found in other surveys (7, 30). In addition, in our study, 79% of published studies originally presented as abstracts had at least one different author than the original presented report, and this rate is higher than the rates observed at other meetings (59-73%) (30, 31). However, the percentage of studies where the first author of the presentation had changed or disappeared in the derived article was lower in our study (18.3%) than in surveys of other analyzed medical specialty meetings (22-36%) (7, 31, 32).

An increase in sample size may be related to the continued enrollment of subjects after presentation of preliminary results, while a decrease in the sample size suggests possible tampering with the data to increase the quality of the paper.

The possible reasons for changes in authorship are more complex and may include further analysis by another investigator, removal of authors if contribution is below standard required for authorship, limiting the number of authors in some meetings or journals, and so called "honorary" authorship. The fact that the first author of the presentation disappeared in 4% of publications is of interest. A possible reason for this result is that this author was chosen only for the presentation.

The present study has some limitations that should be considered when interpreting the results. First, we determined publication status based on a particular medical database (PubMed and Korean medical databases) search; therefore, it is possible that we missed some studies that were published in journals not indexed in the database. Search errors might have been another cause of missing some of the published reports. These errors may occur due to the misspelling of the name of an author or merely because of a faulty search. Although we tried to minimize these errors by searching with two independent investigators, major changes of the author names or the key words from the title stand as possible causes of error during the search. Second, we did not investigate the possibility of duplicate presentations (presentations of the same study at multiple meetings) or duplicate publications (multiple publications resulting from a single abstract) as one of our aims was to compare the publication rate in our study with that of other studies. Only a few previous studies have addressed the situation of duplicate presentation or publication (19, 25). Finally, in this study, no attempt was made to contact the authors of presentations; further research needs to be conducted to determine the precise reasons why presentations were not published.

In conclusion, the publication rate is significantly lower for the KRS (23.6%) as compared to the RSNA (35.4%) and ECR (50.5%) meetings. Prospective design, use of statistical testing, and a positive study outcome have a statistically significant effect on the publication rate.