Exploring the characteristics, global distribution and reasons for retraction of published articles involving human research participants: a literature survey.

AIM: Article retraction is a measure taken by journals or authors where there is evidence of research misconduct or error, redundancy, plagiarism or unethical research. Recently, the retraction of scientific publications has been on the rise. In this survey, we aimed to describe the characteristics and distribution of retracted articles and the reasons for retractions.
METHODS: We searched retracted articles on the PubMed database and Retraction Watch website from 1980 to February 2016. The primary outcomes were the characteristics and distribution of retracted articles and the reasons for retractions. The secondary outcomes included how article retractions were handled by journals and how to improve the journal practices toward article retractions.
RESULTS: We included 1,339 retracted articles. Most retracted articles had six authors or fewer. Article retraction was most common in the USA (26%), Japan (11%) and Germany (10%). The main reasons for article retraction were misconduct (51%, n = 685) and error (14%, n = 193). There were 66% (n = 889) of retracted articles having male senior or corresponding authors. Of the articles retracted after August 2010, 63% (n = 567) retractions were reported on Retraction Watch. Large discrepancies were observed in the ways that different journals handled article retractions. For instance, articles were completely withdrawn from some journals, while in others, articles were still available with no indication of retraction. Likewise, some retraction notices included a detailed account of the events that led to article retraction, while others only consisted of a statement indicating the article retraction.
CONCLUSION: The characteristics, geographic distribution and reasons for retraction of published articles involving human research participants were examined in this survey. More efforts are needed to improve the consistency and transparency of journal practices toward article retractions.

Introduction

Article retraction is defined as the withdrawal of previously published articles, which is a measure taken by journals or authors if there is evidence of misconduct or error, redundancy, plagiarism or unethical research.1 There has been an increase in the retraction of scientific publications in recent years.2–4 For instance, one study showed that over the past 15 years, retractions due to fraud and error had increased by ~1.5 and six times, respectively.2 Even after accounting for publication inflation, a dramatic rise in annual retraction counts persists.3,4 This has raised profound concerns about the quality of scientific work and the social and ethical aspects of research.4

Major reasons for article retraction can be generally grouped into two categories in the literature: 1) scientific error, which includes mistakes in sampling, procedures, or data analysis, and accidental omission of information and 2) misconduct, which includes fraud and plagiarism.5–8 There continues to be substantial debate about whether the increased frequency of retraction is a good or bad sign for biomedical research.9 Some argue that it could indicate increasing incidence of cases of scientific misconduct, while others claim that it could imply an improved system of measures against misconduct, resulting in more cases being identified.2 One study performed a meta-analysis of several surveys that asked researchers whether they had committed or observed misconduct.10 It was found that 2% of scientists admitted having committed research misconduct and one in three scientists admitted having engaged in questionable research practices, ranging from simple carelessness to scientific fraud.10

It is not uncommon that article retractions are not well announced, resulting in the continued citation of retracted articles without acknowledgment of the retraction.11 A blog called “Retraction Watch”, launched by two journalists Ivan Oransky and Adam Marcus in August 2010, examines and reports article retractions and advocates transparency in the retraction processes.11 The blog writers report that article retractions are more frequent than previously thought.11

Some characteristics have been reported on the retracted articles. For example, some studies have shown that retractions appear to be more common in journals with higher impact factors (IFs).8,12,13 It was also found that a positive correlation existed between the male sex of the principal investigator(s) and the likelihood of article retraction.14 Other reported characteristics of article retraction included the lack of reported source of funding in the published paper.5

Despite the significant increase in the number of article retractions in health research, evidence for the characteristics of such phenomenon remains sparse in the literature. In this survey, we aimed to explore the factors associated with article retraction, the global distribution and the reasons for article retraction. We also evaluated how article retractions were handled by journals and how to improve the journal practices toward article retractions.

Methods

Search strategy

We searched PubMed database systematically by 1) using the descriptor “retract*”; 2) limiting results to human, English language, and article type (retracted Publication) and 3) limiting results reported from January 1980 to February 11, 2016. Table S1 lists the detailed search strategy.

Study selection and data extraction

The inclusion criteria comprised retracted publications involving biomedical research on human research participants and indexed by PubMed as “Retracted Publication”. Articles involving plant or animal research, research not directly related to health and articles reinstated after retraction were excluded. Study selection and data extraction were performed by two reviewers (MK and YJ) independently using a selection checklist and data extraction form developed for the purposes of this study. Disagreement was solved by discussion between the two reviewers, and if no consensus could be reached, a third reviewer (GL or MX) was consulted to make a final decision.

Data collected were grouped into three categories determined a priori: publication characteristics, journal characteristics and retraction characteristics. Publication characteristics included number of authors, sex of senior or corresponding author, geographic location, source of funding and conflict of interest reported. Journal characteristics included journal’s IF and eigenfactor score. Eigenfactor score is a measure of the influence of a journal on the scientific community based on the number and origin of incoming citations.15 Eigenfactor scores for all journals add up to 100, and a higher eigenfactor score indicates a journal’s prestige and importance.15 Retraction characteristics included reason for retraction, parties calling for retraction, whether or not the retracted article had an indication of the retraction and whether or not the retraction was reported on Retraction Watch. Reasons for retraction were classified into six categories: 1) misconduct, which included fabrication, falsification, fraud, plagiarism and lack of ethical approval; 2) error, which included mistakes in sampling, data analysis or procedures; 3) irreproducible results, which could be due to error or misconduct; 4) duplicate publication, which could be due to an error by the publisher or misconduct by the authors; 5) author disagreement, which included authorship concerns or lack of consent of all authors and 6) unknown, which included cases where the reason for retraction was not stated in the retraction notice and cases where the retraction notice was unavailable. Each retracted article identified from PubMed was manually searched to assess whether the retraction was reported on Retraction Watch.

Information was collected from the article summary pages on PubMed, the abstracts and the full-texts, where available and accessible. Sex of senior or corresponding author was determined through the authors’ names. If sex could not be inferred from the name, a search of the author’s name and affiliation was performed using Google Images. If sex still could not be determined, it was entered as “Unclear”, in order to avoid assumptions resulting in biased findings.

Statistical analysis

We used descriptive statistics including medians and interquartile ranges for continuous variables and counts and percentages for categorical variables to describe the characteristics of the retractions and the journals. IBM SPSS Statistics, Version 20.0 (IBM Corporation, Armonk, NY, USA) was used to perform statistical analyses. Ethics approval was not required for this study.

Results

Study identification

There were 2,310 records retrieved from PubMed and 13 records retrieved from retraction notices that involved multiple articles. Excluded records included articles involving plant or animal research (n = 443), research not directly related to health (n = 540) and articles reinstated after retraction (n = 1). A total of 1,339 studies were included for analyses in this survey (Figure 1).

Figure 1

Flow diagram showing the study selection.

Publication characteristics

For retracted articles with available and accessible author lists within our study sample, the maximum number of authors on a single article was 21 and the minimum number of authors was one (Figure 2). The median number of authors was 4 (interquartile range: 3–6). More than 66% of the retracted articles had male senior or corresponding authors (Table 1). Table 2 stratifies the reasons for article retractions by sex of the senior or corresponding authors. For all reasons of retraction, the percentage of retracted articles with male senior or corresponding authors was substantially higher than that with female senior or corresponding authors.

Figure 2

Number of retracted articles by number of authors (n = 1,339).

Table 1

Summary of the characteristics of all the retracted articles in the study sample (n = 1,339)

Characteristic	Number of retracted articles (%)
Publication characteristics
Sex of senior or corresponding author
Female	184 (13.7)
Male	889 (66.4)
Both	16 (1.2)
Unclear	250 (18.7)
Source of funding
Not mentioned	676 (50.5)
Mentioned	408 (30.5)
Public	197 (48.3)
Private	71 (17.4)
Public and private	71 (17.4)
No funding received	69 (16.9)
Unknown due to unavailable or inaccessible article text	255 (19.0)
Conflict of interest
Not mentioned	813 (60.7)
Mentioned	340 (25.4)
No conflict of interest declared	306 (90.0)
Conflict of interest declared	34 (10.0)
Unknown	186 (13.9)
Retraction characteristics
Reason for retraction
Misconduct	685 (51.2)
Error	193 (14.4)
Duplicate publication	169 (12.6)
Irreproducible results	13 (1.0)
Author disagreement	18 (1.3)
Unknown	261 (19.5)
Party calling for retractiona
Journal editor(s)	794 (59.3)
Author(s)	380 (28.4)
Publisher	212 (15.8)
Author’s institution	81 (6.0)
Investigating committee	61 (4.6)
Funding agency	1 (0.1)
Retraction policies
Publications in journals with retraction policies	1,141 (85.2)
Publications in journals with no retraction policies	178 (13.3)
Unclearb	20 (1.5)
Journal practices toward retracted articles
Article completely withdrawn	181 (13.5)
Article still available	1,158 (86.5)
Watermark	599 (51.7)
Note	91 (7.9)
Watermark and note	63 (5.4)
Inaccessible article text	10 (0.9)
No indications of retraction	395 (34.1)
Retraction Watch reporting
Articles retracted before August 2010	446 (33.3)
Articles retracted after August 2010	893 (66.7)
Retracted articles not reported on Retraction Watch	326 (36.6)
Retracted articles reported on Retraction Watch	567 (63.4)

Notes:

Percentages were not mutually exclusive since an article could be retracted by more than one of the listed parties.

A retraction policy was unknown due to the journal being discontinued or the journal website being in languages other than English.

Table 2

Reasons for retraction stratified by sex of senior or corresponding author or authors (n = 1,339)a

Reason for retraction	Female (n = 184)	Male (n = 889)	Both (n = 16)	Unknown (n = 250)
Misconduct (n = 685)	77 (11.2)	510 (74.5)	5 (0.7)	93 (13.6)
Error (n = 193)	37 (19.2)	126 (65.3)	1 (0.5)	29 (15.0)
Duplicate publication (n = 169)	24 (14.2)	101 (59.8)	3 (1.8)	41 (24.2)
Irreproducible results (n = 13)	3 (23.0)	10 (77.0)	0 (0)	0 (0)
Author disagreement (n = 18)	7 (38.9)	10 (55.6)	0 (0)	1 (5.5)
Unknown (n = 261)	36 (13.8)	132 (50.6)	7 (2.6)	86 (33.0)

Note:

Results are expressed as counts (percentages) by each reason of retraction.

In terms of geographic distribution, at the continent level, 36% of the retracted articles in our study sample consisted of research conducted in Asia, 32% consisted of research conducted in Europe and 26% consisted of research conducted in North America. At the country level, article retraction was most common in the USA (26%), Japan (11%) and Germany (10%). Of all the articles retracted due to misconduct, 19% were authored in the USA, 17% in Japan and 14% in Germany. Approximately 31% of articles retracted due to scientific error were authored in the USA, 8% were authored in China and 8% were authored in Japan.

Among the articles with information on funding available (n = 408), 48% (n = 197) were publicly funded, 17% (n = 71) were privately funded and 17% (n = 71) were funded through both public and private sources. As regards conflict of interests, there were 25% (n = 340) articles with data on conflicts of interest available, among which 10% (n = 34) declared a conflict of interest (Table 1).

Journal characteristics

A five-year journal IF was available for 1,228 retracted articles. The median IF was 2.94 (interquartile range: 1.81–4.69). Only 0.8% (n = 10) of the articles retracted were published in a journal with an eigenfactor score >1, such as Science, Proceedings of the National Academy of Sciences (PNAS), PLoS One and Nature. The journal PLoS One had the highest eigenfactor score of 1.53, followed by Nature (1.50), PNAS (1.42) and Science (1.22). Table 3 summarizes the journal metrics for the five journals with the highest frequencies of retracted articles. The highest number of retractions (n = 37) occurred in the journal Anesthesia and Analgesia that had a 5-year IF of 3.45 and an eigenfactor score of 0.031. All the five journals had a retraction policy.

Table 3

Characteristics of journals with the top five highest frequencies of retracted articles

Journal name	5-year impact factor	Eigenfactor score	Number of retracted articles, n (%)	Journal retraction policy
Anesthesia and Analgesia	3.454	0.0308	37 (2.8)	Yes
Canadian Journal of Anaesthesia	2.150	0.0080	32 (2.4)	Yes
European Journal of Anaesthesiology	2.423	0.0060	24 (1.8)	Yes
Obstetrics and Gynecology	5.098	0.0482	19 (1.4)	Yes
British Journal of Anaesthesia	4.691	0.0251	18 (1.3)	Yes

Retraction characteristics

In terms of reasons of retractions, more than half (n = 685) of the articles in the study sample were retracted due to misconduct and 14% (n = 193) were retracted due to error (Table 1). For the parties calling for retractions, retraction was done by the journal editor(s) in most articles (59%, n = 794). There were 13% (n = 178) retracted articles published in journals with no retraction policy. For the journal practices toward retracted articles, only 14% (n = 181) of retracted articles were completely withdrawn (Table 1). Of the remaining articles that were still available, 34% (n = 395) had no indication of the retraction, 52% (n = 599) had a watermark indicating that they had been retracted, 8% (n = 91) had a note stating that they had been retracted and 5% (n = 63) had both a note and a watermark (Table 1). Large discrepancies in the announcement about article retraction were also observed. Some journals merely provided a retraction statement without any information on the reason or the party calling for retraction, while other journals included a detailed account of the retraction event and some journals even provided correspondence between author(s) and editorial staff.

There were 446 (33%) articles retracted before August 2010 (when Retraction Watch was launched). Of the articles retracted after August 2010, 63% (n = 567) retractions were reported on Retraction Watch (Table 1).

Discussion

In this literature survey, we examined the characteristics, geographic distribution and reasons for retraction of published articles involving human research participants. We provided data on article retractions from three different angles, including publication, journal and retraction characteristics.

We identified a large number of articles fulfilling our inclusion criteria of retracted articles. It was found that the median number of authors on the retracted publications was 4. The number of authors in published articles may reflect the complexity of biomedical research requiring multi-expertise and collaborative centers to increase sample size and improve generalizability of the study.16,17 Having a large number of researchers may also mitigate scientific errors and omissions and result in better reporting of studies, which may therefore avoid future retraction of such papers.18 In this study, we have found that the majority of retracted articles had six authors or less (Figure 2). It is therefore an important consideration for peer reviewers, editors, publishers and funding agencies to ensure that researcher teams are composed of individuals with the expertise required to conduct, analyze, interpret and report research studies to avoid potential misrepresentations and inaccurate reporting of the findings.

In this study, we found that the majority of retracted articles were authored by men investigators as the senior or corresponding authors (Table 2). This finding may be explained by sex differences in risk-taking behaviors, including greater tendency for men for intellectual risk taking.19 It may also be because there are more men leading research groups and authoring studies than women in biomedical research, yielding the frequency of retraction being proportional to the number of articles published by sex.20–23 Consistent with a previous report,5 we found most of retracted articles had no source of funding or conflict of interest available. Funding information may represent a source of bias in reporting and interpreting the study results. The desire to produce “significant” results may be stronger among researchers who had close relationship between industry, conducted research using their own funds or had personal investments in the results, among others.24 The lack or incomplete reporting of conflict of interest or funding by authors may therefore be seen as an act of misconduct, resulting in a potential retraction of the article by the journal.25 It is therefore essential that sources of funding and conflicts of interests should be clearly documented in published articles. Besides, findings from our study showed that countries with the highest frequencies of article retractions were the USA, Japan and Germany, which was in agreement with a previous study.26 The high frequency of retracted papers in these countries is likely explained by the high volume of scientific publications in these countries. For example, the USA has the highest number of scientific publications worldwide.27 The large number of retracted articles in a country may also reflect regulatory authorities’ practices and stringent reporting standards of research funded publicly. A misconduct by investigators using funding from the National Institutes of Health, for example, may result in severe penalties, including criminal charges and convictions.28 Additionally, variation by geographic location may also represent accessibility of resources for publications in general and also for investigating misconduct leading to a retraction.

In terms of journal characteristics, all the top five journals with the highest number of retracted articles are specialty journals (four related to analgesia/anesthesia and one related to obstetrics and gynecology; Table 3). One previous study found that two of the top 15 authors with the highest number of retracted articles were in the field of anesthesia,29 which may partly explain the findings of our study. Nevertheless, more evidence is needed to further explore the high number of retracted publications in analgesia/anesthesia.

Regarding retraction characteristics, the most common reasons for retraction were misconduct, followed by errors and duplication of publication, which was in line with other studies.30–32 Misconduct can be seen in the conception and design of the study, the implementation of the study methods, the interpretation of the findings or the reporting of the study results in publications.7 Identification of misconduct is the collective responsibility of many individuals and agencies, including the investigators, the local institutions, the ethics boards, the funding agencies, the regulatory authorities, journal editors, publishers and peer reviewers. Despite the presence of many codes of conduct in the scientific field and the potential of harsh penalties for intentional misconduct, it is unfortunate that misconduct continues to occur in the scientific literature. More stringent practices should be put in place to ensure that actions involving misconduct are identified in a timely manner and handled appropriately.

For the journal practice toward retracted articles, we found that some articles were completely withdrawn, while some were still available with no indication of the retraction whatsoever (Table 1). We also observed large discrepancies in retraction notices issued by journals to announce article retraction (Table 1). Such a suboptimal practice toward retracted articles may result in the continued use and citation of invalid and unreliable research evidence. It is therefore important to make the retracted articles transparent, visible and clear to readers to avoid the use of such evidence. We recommend that journals adopt a standardized policy for handling and reporting article retractions in a consistent, transparent and informative manner. In addition, journals may consider setting up an honest-retraction or self-retraction system to allow authors to proactively withdraw their works when they discover serious genuine errors.33,34 Such a system would also help readers easily know whether the retracted articles were due to either a genuine mistake or other deliberately deceptive actions.34 A recent study by Bilbrey et al35 developed a rating system for evaluating the quality of retraction notices. Based on this rating system and recommendations by the Committee on Publication Ethics (COPE),35,36 we put together a list that outlines the important information about the retracted article that should be provided to readers through retraction notices (Box 1). For example, part of the key elements in a retraction notice should include the specific section(s) retracted and the effect of the retracted section(s) on the rest of the article. Such details would provide readers with a clear picture of the retracted article section(s) and their potential consequence on the whole study.

An informative retraction notice should include:

Citation of the article in question as well as any other relevant articles.

Indication of the parties who initiated, issued and/or supported the decision to retract the article.

Statement of whether the authors were contacted regarding the retraction and whether all authors agreed to the retraction.

Clear and explicit statement of the reason(s) for retraction, including whether the retraction was due to either self-retraction or other reason(s).

Clear description of the sections retracted.

Indication of the effect on the rest of the article if only a certain section is retracted.

Statement of the action taken by the journal toward the article (e.g., complete withdrawal of the article, addition of a watermark or a note) and the authors.

Study limitations

The study has some limitations. The major limitation for this literature survey was the lack of a control group, which thus did not allow for the comparison of the observed trends in retracted articles to those in non-retracted articles. Further research using a comparison group of non-retracted articles is needed to observe the comparative trends and to determine the potential factors associated with the likelihood of retractions in a regression model. Likewise, investigation in the role of other factors may be worth considering. For instance, comparing the number of senior or corresponding male authors in the retracted articles with the number of senior or corresponding male authors in general in science, providing statistics adjusting for the ratio of the number of publications relative to the number of inhabitants per country or assessing whether there is an effect of time on retracted articles in different countries or sex, journal IF and journal practices toward retracted publications may be further worthwhile endeavors. Another limitation is that we did not examine the status of globally retracted articles in different fields of medical specialization or by disease. Moreover, only research articles in English language were examined, which could lead to important trends not being accurately captured in the results. No correlation or regression analysis was conducted to explore the association between journal’s IF and the number of retracted articles, because the majority of journals only had one retracted article (n = 703, 57%). Furthermore, we only searched PubMed and probably missed articles that were not indexed in this database. We restricted the inclusion criteria to human studies, which limited the generalizability of our findings to other biomedical research areas.

Conclusion

Article retraction remains a profound issue in the biomedical community. The characteristics, geographic distribution and reasons for retraction of published articles involving human research participants were examined in this survey. More efforts are needed to improve the consistency and transparency of journal practices toward article retractions.

Table S1

Search strategy on PubMed

Step	Query	Records retrieved
1	Search retract*	34,000
2	Search retract* Filters: Humans	18,448
3	Search retract* Filters: Humans; English	15,608
4	Search retract* Filters: Humans; English; Retracted Publication; Publication date from 1980/1/1 to 2016/02/11	2,310