What is the future of peer review? Why is there fraud in science? Is plagiarism out of control? Why do scientists do bad things? Is it all a case of: "all that is necessary for the triumph of evil is that good men do nothing"?

Peer review is an essential component of the process that is universally applied prior to the acceptance of a manuscript, grant or other scholarly work. Most of us willingly accept the responsibilities that come with being a reviewer but how comfortable are we with the process? Peer review is open to abuse but how should it be policed and can it be improved? A bad peer review process can inadvertently ruin an individual's career, but are there penalties for policing a reviewer who deliberately sabotages a manuscript or grant? Science has received an increasingly tainted name because of recent high profile cases of alleged scientific misconduct. Once considered the results of work stress or a temporary mental health problem, scientific misconduct is increasingly being reported and proved to be a repeat offence. How should scientific misconduct be handled--is it a criminal offence and subject to national or international law? Similarly plagiarism is an ever-increasing concern whether at the level of the student or a university president. Are the existing laws tough enough? These issues, with appropriate examples, are dealt with in this review.

Dear Editor: Do you have a problem?

“All that is necessary for the triumph of evil is that good men do nothing.” This common quote is attributed to Edmund Burke, born in Dublin in 1728, died in Beaconsfield, Buckinghamshire (on 8 July) 1797. The use of this quotation is so common that it is rare that one even bothers to acknowledge Mr. Burke or the text from which this quote has been supposedly extracted—too bad because, no doubt, this would be, by today’s classification, a citation classic. With web searching so easy it should be easy to locate the precise source of a quotation. Are we then guilty of plagiarism? Have the Editors of this journal failed in their task of due diligence in accepting this review article? Will our institutions be brought to task for this transgression? Should there be a national or international body to deal with such matters? In this article the brothers Triggle discuss these and other matters related to the integrity of science and offer their opinion as to the future of peer review. They bring over 85 years of academic experience, over 400 peer reviewed manuscripts (including a number jointly authored, including one book) and over 6000 citations for their collective works.

Why do we need peer review?

Peer review should provide due diligence to a manuscript or grant and this requires a considerable time commitment. This is not new: peer review has a long history, predating even the review process of the Philosophical Transactions of the Royal Society, initiated in 1752. For the first 100 years of the journal’s existence decisions on publication were the responsibility of the editor alone and those of his colleagues whom he might have asked (Eaton 1997; Spier 2002). With the very rapid expansion in both the numbers of journals as well as sub-specialities it is now, of course, unreasonable to expect any single editor to possess the vision and depth of knowledge to be competent in all areas. With many journal reviews now “on line” the peer review process is presumably more efficient, with the attendant potential cost of an increase in the number of requests that the efficient reviewer receives. Peer review is widely, and perhaps almost universally, regarded to be an essential component of the scientific review process and to provide quality control so that the published works meet appropriate standards. Most of us would agree, at least in public, that peer review works reasonably well, but it is certainly not without its problems and the issue is what could be better or constitute improvements. An important first question is to define “peer”? One definition (from The Concise Oxford Dictionary) is: “A person who is equal in ability, standing, rank or value.” Is it then ethical for a peer to pass a review on to a junior postdoctoral fellow? The junior fellow requires developing experience as a reviewer but should that be obtained by reviewing manuscripts/grants that were originally directed elsewhere? We all have personal views on the fairness of peer review. Winston Churchill’s comment on democracy comes to mind, “…democracy is the worst form of government except all the others that have been tried”. However, that being said, like everything else the peer review process should evolve—the question is: “In what direction and should the peer review process actually police scientific fraud and should the peer review itself be subjected to review and potential legal action if scientific fraud by the reviewer is suspected?” (see Ready 2006). This topic is the focus of much recent discussion in both the scientific and public press (inter alia, Altman 2006; Bosman 2006; Couzin and Unger 2006; McCook 2006; Marris 2006). Nature Medicine, in its May 2005 2006 issue, ran a series of commentaries titled “Focus On Fraud” that reflect the high level of concern that is being focused on matters of scientific integrity.

“My gut feeling is that this will not work!” “Yes, but please justify that statement.” Does anonymity lead to laziness?

The argument for reviewers remaining anonymous is that they are then protected from retribution from a potentially irate author(s). Furthermore, the pool of available reviewers may dramatically decline if the names of all reviewers were published. This all assumes, of course, that maximal effort and fair judgment is provided to every submitted manuscript and this, unfortunately, is not always the case. Not infrequently reviewers will include un-qualified statements such as, in an extreme case, “My gut feeling is that this is incorrect”, in their review that may, in fact, itself be a scientifically incorrect statement! Such useless statements then leave the journal editor, or grant panel chair, with the task of evaluating and rejecting the reviewer’s review. Worse, though is that such an incompetent review may lead to the rejection of the submitted paper, or of the grant application, and the ultimate failure of the career of the author. Could this happen? Yes, indeed and we will discuss an example later when a Texan general practitioner decided to fight back after he argued that he was victimized by the peer review process. To their credit many agencies do make every effort to weed out the inaccurate review and, in fact, vigorously review and rate the reviews, rejecting some reviews, as well as developing preferred lists of reliable reviewers. Without doubt the review process could be greatly improved and the process fairer if reviewers substantiated their statements with appropriate references to peer reviewed articles that, in turn, provide positive feed back to the authors. In other words consider the review itself as a scholarly document. Just as a manuscript may be rejected, or a grant receives a low score, if the authors fail to demonstrate/indicate knowledge of the field then should their review be rejected if it fails to meet minimal standards?

So why not publish the reviews?

If the review itself is to be a scholarly document then why not publish such reviews together with the suitably revised reviewed manuscript, perhaps just highlighting in the review the key controversial aspects and presenting the reviewed paper in the perspective of the published field of knowledge? If the reviewer goes to the trouble of providing a critical review that discusses both the key findings as well as the limitations of the study then surely this warrants the equivalent of a “Letter to the Editor”.

Letters to the editor are already a feature of many journals so why not extend such a process by including a section dealing with “Highlights from Reviews of Published Manuscripts”—another section for your CV and your institution to evaluate during promotion and tenure considerations. A number ofjournals, including Nature and Science, do after all provide News and Views columns that could easily be extended.

A strong case for publishing reviews, or criticisms, of a published paper has been made by Eaton (1997). Eaton (1997) focused on “position papers or statements” that were intended to influence medical practice, but, in some instances, such position papers may have inappropriately dismissed alternative approaches/views. To quote Eaton (1997): “Medical science can only flourish in a free society and dies under totalitarian repression.” Of interest is that in the fictional work ‘State of Fear’, by Michael Crichton (2004), the millionaire philanthropist, George Morton, also argues in favor of the publication of both the article and the peer reviews in the same issue as a means for “clearing up everybody’s act real fast”.

Of critical importance is that the journal Nature has just launched Nature Peer Review Trial and Database that may well lead to a revolution in the peer review process and would appear to address at least some of the concerns regarding transparency of the review process. Other journals should consider following this lead.

Are you too positive?

What about negative data? There is a tendency for editors and reviewers to only accept so-called “positive data”, but there is also the need to publish data that, although negative, may still help advance the field. How best to do that? Should there be a publicly accessible depository for negative studies? Scientists often do not dwell on their negative data, but by (trying) to ignore these “failures” are they also being unethical? Perhaps including a description of protocols that did not work together with data from those that were “successful” in a publication is a more honest approach—but will that paper be favorably reviewed and would the journal accept this when page restrictions apply? Whatever the approach we do argue that a process whereby so-called negative data can be made available is required.

Are changes in the peer review system essential?

Evidence that bias may enter the review process, at least for abstract submissions, was provided by an analysis by Ross et al (2006) of the 67 275 abstracts submitted to the American Heart Association (AHA) over the 5-year period 2000–2004. During the period 2000 and 2001 authors names and origins were included in the review process; however, for 2002, 2003, and 2004 the abstracts were reviewed anonymously and the data suggested that well-known laboratories may get a relatively free passage and that the country of origin may also influence acceptance.

If bias exists in the review of scientific abstracts then almost certainly bias exists at the level of manuscript or grant review, but what can be done? Ross’s study may establish the viability and value of a blinded peer review process for the acceptance of abstracts at scientific meetings, but will it work for full manuscripts to journals or for grant applications? Some difficulties with the universal adoption of this process are, however, obvious. For instance, how will an author maintain anonymity and, at the same time, reference “previous work from our laboratory?” Another challenge for grant reviews will, of course, be “evaluation of track record”. Nonetheless, efforts do need to be made to reduce the suspected bias that may benefit some and negatively impact others in the peer review of manuscripts and grants—based on the AHA study it is almost certain that author and institution bias also exists in the review of manuscripts and grants. Such a “halo” effect has long been recognized (Thorndike 1920) and certainly applies to many fields outside scientific publishing and research. Assessment of bias (Gilbert 2006) is an important task for the Editor/Committee Chair, but is also very time consuming and how much additional work should be added to that already burdening most editors?

A step towards establishing an international forum on such issues was made with the establishment of the Committee on Publication Ethics (COPE) in 1997. COPE has a current membership with editorial representation from 346 peer-reviewed journals and the mandate to discuss issues related to the scientific integrity of the publication process. To date COPE has published seven reports (see and McCall Smith et al 2000). The establishment of COPE is certainly a step in the right direction, as journal editors clearly need support and guidance as to how best deal with suspected/alleged scientific fraud (see Jones 1999). COPE was chaired by the editor of the BMJ, Fiona Godlee, for the period 2003–2005 and the BMJ, itself has been the centre of attention in 2005 discussing allegations concerning scientific integrity against two prominent scientists—see below—“Publish and then perish—fraud in science and the case of the repeat offender.”

“Thank you, but your (bad) review just cost me my job and I’m suing you”

In 2000 Dr Schulze won a settlement of close to US$15 million after the court agreed that he had indeed been victimized and his reputation severely damaged by a badly conducted peer review of his medical practice by a Health Maintenance Organization (HMO) (Rice 2001). Prior to the HMO’s peer review Dr Schulze, a general practitioner from Corpus Christi, Texas, held an excellent reputation of untarnished medical practice spanning 35 years that suffered during a period of investigation by his peers that lasted approximately 6 years. The lawyer for the HMO argued that Dr Schulze’s victory was a set back that damaged the confidentiality of the peer review process, which is an essential component of the process for maintaining the quality of health care. The fact is, however, that if the peer review process is unfair, if the rights of the individual under review are not protected, if the “facts” presented during the review are inaccurate, and the result is a damaged reputation and loss of income then why shouldn’t you sue? We are, of course, not recommending that everyone who believes that the review of their grant or manuscript was conducted by an incompetent or vindictive reviewer launch a law suit, but the case of Dr Schulze reveals that the process of peer review is a very serious matter and must be conducted fairly. The onus, of course, lies with the committee chairs and editors to be vigilant and recognise what might be considered unfair or bias in the peer review process, but it is up to all of us, as the reviewers, to make their task easier by being fair and commit the time to what is a very important (but usually underappreciated and unpaid) job. Reviews that either intentionally or simply due to laziness and/or incompetence misrepresent what the authors have stated also reflect scientific fraud or misconduct by the reviewer. In other professions incompetence usually results in penalties. Why not the same for incompetent or fraudulent reviewers? The solution, of course, may well be a few well-aimed lawsuits that will wake up the scientific community from its complacency; however, is that really what we want and would this destroy the peer review system? A better solution is to make the review process more open and accountable.

Who should peer review your research—the FBI or a magician?

A strange choice, but the FBI and a professional magician have been used to assist the peer review process and in both cases, not surprisingly, the results resemble a witch-hunt.

For Dr Mark Feldstein, formerly an investigative reporter with CNN and now with George Washington University as Director of The Journalism Oral History Project, it was the FBI who visited with him to discuss their interest in the research work that Dr Feldstein was pursuing on the late Jack Anderson (Feldstein 2006). Jack Anderson, who died in late 2005, was described by Henry Kissinger as “the most dangerous man in America” (others have, of course, described Dr. Kissinger, with at least equal justification, in similarly unflattering terms [Hitches 2001]), but he was also the recipient of the 1972 Pulitzer Prize for National Reporting and his career was dedicated to uncovering corruption with, as examples, J Edgar Hoover, Watergate, the JFK assassination, the Iran-Contra affair receiving his attention. It would not be surprising therefore if the FBI were to either recommend “accept only after major revisions”, or “rejection”, of any forthcoming publication that focuses on information obtained from the files of this controversial figure.

For Dr Jacques Benveniste, discoverer of platelet activating factor (PAF) in 1970 and highly respected INSERM scientist, it was his decision in 1988 to pursue publication with colleague Dr Bernard Poitevin of data arising from an allergen high-dilution “memory of water” supporting the concept of homeopathy that resulted in a visit from a magician (Davenas et al 1888; Editorial Opinion 1988). Dr Benveniste submitted the paper to Nature and acceptance came with the proviso that the then editor, Dr John Maddox, be allowed to send an investigative team to visit Dr Benveniste’s laboratory and view the studies first hand (see Benveniste 1988a, 1988b, 1988c; Maddox 1988 for correspondence regarding the review process). The make up of the investigative team, however, reflected the extreme skepticism of Dr Maddox and included a professional magician and a journalist intent on exposing fraud or, at least misinterpretation, which, arguably, they did (Maddox et al 1988). Not surprisingly perhaps Dr Benveniste’s career took a nosedive as the French scientific community felt that French science had been dishonoured, and his laboratory ultimately closed. Dr Benveniste’s honour, however, was at least partially restored with the success of his own company, Digibio (), as well as a publication that seemingly supports, at least in part, the conclusions from his 1998 paper in Nature (Brown and Ennis 2001)—a view he himself maintained. “Why then accept a paper on 13th June to publish June 30th to destroy on 8th July data so easily spotted as wrong or made up?” (Benveniste 1988a).

Readers will agree that neither of these examples speak well of the peer review process.

Is this really a conflict of interest—it never occurred to me?

The real challenge is whether to declare the conflict or to avoid it in real life and to avoid being on the front page of your hometown newspaper. One gains no credibility by declaring that one is the reviewer of one’s own manuscript, the only acceptable solution is not to review it. Similarly a grant review panellist should not serve on a panel where their own (or a grant from a colleague or collaborator) grant is before the panel, but, very surprisingly, this is still common practice with some granting agencies and yet is not recognized as a conflict—how can this be? No wonder there is scepticism about some of the decisions made in some countries by the funding agencies. Similar concerns can be raised at the university level. Can you really expect no matter how well structured an institution to police its own policies and impartially investigate questions of scientific integrity (see Smith 2005)? There clearly is an urgent need for appropriate policing bodies at the national or even international level—see section below on “Fraud and discussion of the role of the Office of Research Integrity. The case for national/international monitoring and adjudication is very strong as how many individual institutions have enough experience and expertise to adequately respond to allegations of scientific misconduct?” The answer is few—if any.

Of course, conflicts are not only with the author and/or reviewer. They can also exist at the level of the editor, the editorial board, and the publisher. Most recently, the editor of the Canadian Medical Association Journal (in 2006 ranked as the fifth leading general medical journal in the world) was dismissed by the publisher, apparently for publishing articles dealing with marijuana and emergency contraception that did not accord with the views of the Canadian Medical Association, the journal’s owner (Shuchman and Redelmeier 2006). Ironically enough, the editor of the journal, John Hoey, had previously published editorials on the similarly politically influenced dismissals of George Lundberg and Jerome Kassirer as editors of the Journal of the American Medical Association and the New England Journal of Medicine respectively (Hoey 1999; Hoey et al 1999). Bringing public attention to issues of “scientific integrity” may lead to the adoption of guidelines and the resolution of the problem. With respect to “editor censorship” and the Canadian Medical Association Journal this now seems to be the case following a recommendation by an independent committee established to resolve the question of “editorial independence that the mission statement of the Canadian Medical Association Journal be amended to: “the principle of editorial integrity, independent of any special interests” (Birchard 2006). Decisions by the editor can also generate conflict. Thus, when Nature concluded that a previously published paper describing the occurrence of transgenic DNA in Mexican corn (Quist and Chapela 2001) should not have been published, “Nature has concluded that the evidence available is not sufficient to justify the publication of the original paper” (Editorial comment 2002) issues were immediately raised as to the appropriateness of both the original peer review process and the subsequent scientific comments leading to Nature’s decision and as to whether the editorial decision was appropriate, raising the question of what is appropriate or inappropriate at the level of an editor’s decision concerning the submission/review of a manuscript (see also the reference to Dr Jacques Benveniste—a case that we have already discussed.

Furthermore, the decision fueled the ongoing debate about the role of agricultural biotechnology companies and their relationship to the University of California and to the original decision by the university not to grant tenure to Ignacio Chapela. Science loses its intrinsic claims to truth and objectivity with events like this. More recently, the editor of Cell, Emilie Marcus, retracted a widely noted paper from Brazilian scientists that had claimed that the parasite responsible for Chagas disease inserted DNA into the host genome (Nitz et al 2004), on the basis that following, “careful and extensive review by independent experts…do not provide strong support for the central hypothesis and are open to alternative interpretations” (Retraction 2005). The paper now appears online marked with the word “RETRACTED” in red. Both of these decisions by editors raise important questions about the peer review process, and how the papers originally passed muster; in the absence of fraud would it not be better to simply let the scientific debate play out in print or online. After all, as Richard Feynman famously noted, uncertainty is a key feature of scientific discovery (Feynman 1988).

Conflicts in the peer review process can also be political or religious, derived from some vested interest or ideology whose interests are threatened. The decision by the US Food and Drug Administration (FDA) not to approve over the-counter availability of a post-coital contraceptive pill (“Plan B”) despite the approval by its scientific advisory board, a decision that the Government Accountability Office (GAO) itself described as “unusual” (Government Accountability Office 2006) is certainly linked to the present Bush administration support for and by the powerful “right-to-life” community (Drazen et al 2004; Davidoff 2006). Not coincidentally, the FDA announced a possible resolution of the issue on the very day that the nominee for the FDA Directorships, Andrew von Eschenbach, was to appear before Congress at a confirmation hearing (Saul 2006). More recently, a paper published in Science by an Oregon State University student, Daniell Donato, arguing that salvage logging post-forest fire might be detrimental rather than beneficial (Donato et al 2006), was challenged prior to publication by faculty members from that institution, reportedly on the basis that its publication would offend the logging industry in Oregon (Brainard 2006). Fortunately for both the causes of integrity and peer review the editor of Science declined to delay publication (Kennedy 2006). In addition, to the credit of Oregon State University (OSU), the Provost and the Chair of the Faculty Senate of OSU came out with a strong statement defending academic freedom. Conflicts in the peer review process also occur through the vested interests of governments. The American Association of Petroleum Geologists rejected two papers for publication post acceptance for publication because of a US Government policy prohibiting publication from countries under trade embargo. In these cases one of the authors worked for the National Iranian Oil Company and in the other the paper’s authors in Norway had obtained data from the oil company (Guterman 2006; Gripsrud 2006). Both articles will be published elsewhere so you have to wonder what has been accomplished by this attempt at censorship, a phenomenon that appears to be on the rise in an increasingly xenophobic United States.

Finally, conflicts appear between the authors, the journal and the funding source. These typically appear when the funding source (almost without exception of commercial origin) wants to delay or even prohibit publication altogether, in the latter case typically because the published paper would not be favorable. Such cases have been discussed extensively elsewhere (Krimsky 2003; Shuchman 2005; Triggle 2005a) and are less a reflection of the peer review process than of the failure to eliminate the conflicts of interest before the research is initiated.

Having a problem publishing your paper? No worries—just launch your own journal!

Of course, you will need a wealthy backer and, if you accept the views put forward in an article in The Lancet (Garne et al 2005), the tobacco industry may have provided backing and undue influence in establishing the research journal Indoor and Built Environment. Garne et al (2005) report that, since its birth in 1987, Indoor and Built Environment has published a surprisingly large percentage of manuscripts from authors having tobacco industry connections that reflected a favorable view on the health effects of environmental tobacco smoke. Such revelations only add fuel to the fires that are flaring up globally concerning scientific integrity and what really is “good science”. According to many, good science is that published in journals that have a high impact, but is this true or just another urban legend reflecting unsubstantiated beliefs maintained for self-perpetuation of the scientific elite?

Publish or perish or publish and then perish?—The real meaning of JIF

The emergence of thejournal impact factor or JIF has greatly influenced how we evaluate science. The argument in favor of “impact factors” was first mentioned some 50 years ago (Garfield 1955) and its history and meaning reviewed recently (Garfield 1999, 2006). Thus, JIF was originally proposed as a measure for selection and inclusion of a journal in Science Citation Index (Garfield 1999, 2006), but has become extensively used as a means of defining the impact of a scientist’s research and, indeed, an individual’s career and the Institute of Scientific Information’s (ISI) Journal Impact Factor has served as the cornerstone for categorizing journals for approaching 50 years. The number of citations for an article in a given year, the numerator, and the denominator determines the JIF, which is the number of articles/reviews published in the samejournal during the past two years. Most evaluators misunderstand the true meaning of JIF and incorrectly assume/infer that a publication in a journal with a high JIF must have a high impact. This is far from the truth as we will indicate. This confusion is, perhaps, not surprising as, according to the science fiction novel, Hitchhikers Guide to The Universe, by Douglas Adams, the second greatest computer of all time, “Deep Thought” took 7.5 million years to determine that 42 was the ultimate answer, but what was the ultimate question? Similarly, to provide a numerical value to the ‘impact’ of a scientist’s publications is of questionable significance. What does it really mean when a reviewer states: “Dr X publishes in high impact journals?” Does this really imply that Dr X’s publications also have a high impact? Careers for scientists are made or lost based on an individual’s track record—publish frequently and in high impact journals or suffer the consequences. Is this fair? The argument in favor of such a draconian approach to career selection and progression is, of course, Darwinian. During the past 40 years there has been an increasing attention paid to where your paper is published and, of course, we all believe, or, at least, hope that our data are worthy of a paper in high impact (JIF) journals such as Nature or Science. The benefit to the authors of publishing in a high profilejournal is the anticipation that their article will have greater visibility and, therefore, more likely to be cited. Such benefit also contributes to the pressure to obtain results and to publish, a pressure that is not necessarily always beneficial to science or to the scientist. This issue is well presented in a recent novel, “Intuition” (Goodman 2006), set in an active and competitive research laboratory in the Boston area.

So what is the answer? Generally it is the subscriptions to the higher impact journals that libraries will purchase and it is this same group of journals that specialists are most likely to peruse. But the “80:20” phenomenon indicates that 20% of publications accounts for 80% of the citations (see Garfield 2006). Whither the others? It has been stated that during the period 1900–2005, 38 million articles were published, but only 0.5% of these have been cited more than 200 times and half were not cited at all (Garfield 2006). One can only speculate as to why the authors of these 19 million uncited papers did not consider citing these papers in subsequent publications. 19 million peer-reviewed papers that have never been cited—does this imply that this immense amount of research effort was all in vain? Does a paper with zero citations after, say, 5 years imply that this publication had zero impact on the research field and that this research was entirely devoid of impact? Whatever your views it should be apparent that, to coin a popular phrase, “the proof is in the pudding”. In other words it is essential to evaluate the impact of the individual paper and take into account not only where it was published but, in particular, also how well it has been cited and by whom (a process that can be readily accomplished by access to ISI Web of Science)—a paper in a high impact journal does not necessarily equate with a high impact paper, it is the citation frequency that is more important. As stated by Seglen (1997): “Article citation rates determine thejournal impact factor, not vice versa,” and “JIF correlates poorly with actual citations of individual authors.”

The views of Seglen have also been referenced by Garfield (2001). Despite these serious concerns about the misuse of JIFs it is still common practice for reviewers to refer only to the JIF and not consider the content and impact of the individual paper although, of course, it may be argued that for a recently published article insufficient time has elapsed to determine an impact. Other considerations to bear in mind when using the JIF is the problem of padding the citation frequency by self-citations—a process facilitated by journals that have the advantage of a rapid-review and e-pub process and exacerbated by multi-authored papers wherein, each individual author may cite the paper in a subsequent publication—perhaps leading to several citations of the one piece of work in one year by the same group of authors! So, when it comes to the funding of research where should the money go? Should it go an individual, or group of individuals, that publishes solely in high impact journals, but with limited (perhaps self) citations, or should it go to a project from scientists publishing in less prestigious journals but with frequent citations by their peers? Purists might argue that the number of citations is irrelevant and that publishing in the “best”journals is the chief criteria for funding, but then what is the meaning and relevance of “impact”? A potentially more useful index of individual productivity, “h”, has been proposed by Hirsch (2005) where h is defined as the number of papers with citation number >h. With the increasing impact of web-based publications alternatives to the ISI JIF ratings should also be considered. Bollen et al (2005) have also argued that, in part because of the emerging impact of web-based publications that are not included in ISI’s selected list (a point also raised by others), the JIF does not provide an accurate assessment of the true impact of the published article. These considerations thus necessitate that we should look at other means of assessment, such as web hits and downloads that can provide additional data to that obtained via JIF and associated citations for assessing impact.

In conclusion, if you must use JIF then first you must understand what it really means and then use it appropriately and fairly and also seek other parameters to assess the impact of the author(s) research as well as the paper/grant that is being assessed.

Fraud in science and the case of the repeat offender

The difficulty with allegations of scientific fraud is the need to determine with complete certainty that malicious intent and not interpretation error, or simply bad laboratory practice, was the cause. Fraud in science, at least where it has been discovered, has often been explained as reflecting the misdirected activities of an individual suffering from excessive stress—perhaps a post-doctoral fellow who is anxious to ensure a successful career progression. Furthermore, some well-known instances of fraud are often referred to as hoaxes thus suggesting that the intention behind the offence was simply that of an innocent prank. A famous case is the Piltdown forgery—in 1913 a skull was discovered at the Piltdown archaeological site in England that seemingly had similarities to a human cranium and an ape’s jaw thus fitting the expectation of the day that brain size increased first in the evolution from ape to modern man and was the driving force for this change. Additionally, the discovery probably fitted also into the political-social climate of the time—given the then still significant power and influence of the British Empire: what more natural that this “dawn man” should be British? However, in 1953 the find was exposed as a “hoax” and the skull revealed to be made up from the cranium case of a modern man and the jaw of an Orang Utan. Speculation, however, still remains as to whom was responsible although it has been argued that the perpetrator was Martin Hinton, the curator of Zoology at the London Natural History Museum (Gee 1996). Possibly the Piltdown hoax was originally fabricated as a joke, but the truth took 40 years to emerge and confused both the literature on the evolution of hominids as well as many physical anthropologists (Walsh 1996). Another notable and more recent case is that of Dr William Summerlin who, in the early 1970s, was a scientist working on organ transplants at the Sloan-Kettering Institute for Cancer Research. Dr Summerlin was discovered to have used a black felt-tip pen to enhance evidence for the success of grafts of black skin grafts from black onto white mice. Investigations revealed that earlier data concerning the success of human cornea transplants into rabbits was also suspect. Using a black felt-tip pen to falsify data would be considered rather amateurish today given the potential for the use of computer-assisted manipulation. Unfortunately there are many other cases of scientific fraud indicating that the problem is more common than should be expected (see Lock et al 2004). However, it must be admitted that it is, in most cases, impossible for the journal to detect scientific fraud.

It may be argued that scientific fraud set in the framework of a particular political ideology is a particularly dangerous event. An obvious example is that of Trofim Lysenko whose influence on Soviet agriculture was supported by Joseph Stalin and contributed significantly to the massive starvation in the Soviet Union in the 1930s (Graham 1993). Today, and paradoxically enough in the United States, we see a considerable influence of political and religious ideologies on science policy driven enthusiastically by the Bush administration (Mooney 2005; Triggle 2005b). Two recent examples of such conflicts were discussed in a previous section (“Is this really a conflict of interest it never occurred to me?”)

Rather than simply viewing scientific fraud as the isolated lapse of an otherwise honest scientist, as was the argument with Dr Summerlin and, initially, with Dr John Darsee, the current view is that many cases of scientific fraud really reflect repeat offenders. As an example, Dr Darsee, a young cardiologist and NIH fellow, was discovered falsifying data while at Harvard in 1981. It was ultimately revealed that he had been falsifying data for many years at several (at least three) institutions and the NIH, through the NHLBI, launched an investigation (see Culliton 1983). The Darsee case led to revealing questions concerning the role and responsibilities of the co-authors of Darsee’s published papers, but, as it happens, the co-authors were unaware of any falsification with the argument, perhaps, being that Darsee’s “success” may have clouded their judgment that, after all, (initially) reflected well on his mentors and the institutions. Questions were also raised about the level of supervision by Dr Darsee’s mentors. Of particular interest and concern was that a report on the “Darsee affair” (Stewart and Feder 1987) may itself have been flawed (Braunwald 1987). This case thus also stresses the importance of a fair peer review process to investigate the extent of the alleged fraud (Nature opinion article 1987). Concerns on the pressures placed on postdocs to boost their publication record to obtain fellowship support or their first faculty position and/or research grant are a major concern (Benderly 2006).

Scientific fraud is now beginning to be seen as no different from any other criminal and often perpetrated by a repeat offender. A US-based survey suggests that the incidence of falsification, fabrication and plagiarism is higher than one would have hoped with approximately 33% of the participants admitting to one or more of the top 10 (mis) behaviours (Martinson et al 2005). So where should behaviour modification begin? Presumably such modification should start at the top with national governments and academic institutions establishing policies that are both followed and enforced.

Indeed, fraud in science, whether initially intended as hoaxes or planned with career and profit-making intentions, not only ruins the careers of the perpetrator, but also, potentially, their innocent colleagues, as well as tarnishing the reputation of the institution where the work was performed and reducing the confidence of the public in the value of scientific research. Fraud in health research may also have direct or indirect negative effects on health care and peoples lives; fraud in other areas of science may, of course, affect the economy and the lives of people. It can been argued that one approach to dealing with scientific fraud is to proceed through civil courts as, likely, misuse of grant funds is also involved (see Smith 2005).

Several recent instances of alleged fraud in nutrition research—Drs Ram B Singh of India and RK Chandra from Canada (see White 2005; Smith 2005, respectively), in cancer research with Dr Jon Sudbø of Norway, and, in the area of stem cell research, Dr Woo Suk Hwang of South Korea have been reported. Interestingly, all of these scientists achieved almost hero status in their own countries: achieving such status may be a significant motivation for scientific fraud.

An analogy is the athlete who receives the gold medal at the Olympics but then has it stripped from them (and their country) as the result of a drug-tainted urine sample. In the case of Dr Hwang the rise and fall were both dramatic and fast with his landmark paper on stem cells from a cloned human blastocyst (embryo) first published in Science in March 2004, a second paper in Science in June 2005 and both withdrawn in an editorial retraction in January, 2006 (Hwang et al 2004, 2005; Kennedy 2006). The only good news in this the Hwang case is that his Afghan cloned puppy, “Snuppy”, was, apparently, real (Lee et al 2005; Lee and Park 2006). The negative fall-out for stem cell research internationally has affected not only Dr Hwang’s US-Based collaborators but, by creating public and scientific anger and dismay, will likely significantly slow progress in this entire field. The impact of the other allegations noted above in terms of nutrition and human health are, potentially, no less severe. The reaction of the Norwegian government to the fraudulent study by Dr Sudbø (Morris 2006) has been strong, and may lead to the government passing legislation that will make medical fraud a criminal offence that probably should be extended to all areas of science.

Who should investigate allegations of scientific misconduct? Arguably journal editors may be the first to become informed of such allegations, but is it their responsibility to pursue or the institution(s) where the research was pursued? We have argued already that certain responses from editorial offices may be inappropriate (re The Benveniste case) so perhaps it is best that the institution(s) where the research was conducted should first investigate the allegation(s). This appears to make sense, but, in the case of Dr Chandra, proved difficult. Furthermore, with scientific research frequently involving multiple centres and more than one country the argument for an international authority to deal with issues of scientific integrity is strengthened (see also White 2005; Smith 2005). However, institutions and journals should perhaps be the repositories for the electronic data submitted together with manuscripts for publication purposes? Adopting such a system would not prevent publication but would provide a fall-back check system. There are national requirements for maintaining research records (in the United States 3 years from the termination of the grant and 2–4 years from the termination of the contract. In Canada a 7-year period is required. In Australia the recommendations are 5 years from the date of publication, but in some areas, such as for clinical trials a minimum of 15 years is recommended). An external data repository would eliminate the potential for those accused of data fraud of arguing that the records were lost (apparently eaten by termites in the case of Dr Singh—see White 2005). Data repositories should not hold up or prevent publication but would provide a fall-back check system. Such data repositories exist, of course, for the storage of X-ray crystallographic data and nucleic acid and amino acid sequences and almost without exception deposition of these data is a prerequisite for journal publication.

The Office of Research Integrity, or ORI, evolved from the Office of Scientific Integrity that was established in 1989 in response to an increasing number of allegations of scientific misconduct in the areas of biomedical and behavioural research in the USA. The ORI, officially launched in 1993 and independent from the NIH, has as its mandate the promotion of scientific integrity at the international level and holds conferences and issues reports (see ). The ORI is located in and supported by the Office of Public Health and Sciences in the Department of Health and Human Services in Rockville, Maryland, USA. A panel to oversee research integrity in biomedical research has been formed in the United Kingdom. A recent paper by a member of the ORI (Pascal, 2006) outlines some of the issues that the ORI faces in investigating allegations and, of particular interest, are the rights of the complainant, the accused and the interests of the institution. The potential of retaliation against the complainant is a major issue that may well prevent more cases of fraud being uncovered and clearly needs attention.

The UK Panel for Research Integrity in Health and Biomedical Sciences was launched on April 12, 2007: some concern as to its independence has beenjustifiably expressed since the panel will accept money from the pharmaceutical industry (Giles 2005). Regardless of issues of independence we argue that the establishment of national offices for the investigation of questions of scientific integrity is an important step forward as part of an international approach to dealing with this criminal activity.

An indication that scientific fraud is being taken more seriously is the case of Dr Eric Poehlman of Vermont and his sentencing by the federal government for falsifying data for 17 grant applications (see “Focus on Fraud”, Nature Medicine, May, 2006). The prosecution of Dr Poehlman resulted from the combined efforts of the US Attorney’s Office for the District of Vermont, the ORI and the Office of the Inspector General and provides an example for other jurisdictions and countries to follow (see Dahlberg and Mahler 2006; Pascal 2006; Sox and Rennie 2006).

Plagiarize

Defined in the Concise Oxford Dictionary as: “1. take and use (the thoughts, writings, inventions, etc. of another person) as one’s own. 2. pass off the thoughts etc. of (another person) as one’s own.” This constitutes intellectual theft and documentation of its occurrence in academia can be traced back at least 200 years with a clear case of plagiarism reported by Dr Baumes and concerning a thesis submitted by a Francois Bidault to the University of Paris in 1804. Plagiarism today is apparently rampant at the level of high school and university students: according to Donald McCabe Founder and President of the Center for Academic Integrity (), on most campuses 70% of students admit to some cheating (Campbell 2006). Warnings to students about plagiarism receive high visibility in academic institutions, but how seriously are such policies policed? Universities may be reluctant to proceed to formal dismissal when high student quotas are required to maintain government grants and/or tuition revenue.

Plagiarism is likely also a problem with academic publication, although precise data are not readily available (Martinson et al 2005; Nature special report 2005). Indeed, the impact of plagiarism was put to song by Tom Lehrer (a satirist of the 1950s and mathematician and former Harvard student and teacher) about the Russian mathematician Nicolai Ivanovich Lobachevsky (the name apparently chosen for rhyming purposes and not necessarily to imply that Lobachevsky was guilty of plagiarism!):

“Plagiarize,

Let no one else’s work evade your eyes,

Remember why the good Lord made your eyes,

So don’t shade your eyes,

But plagiarize, plagiarize, plagiarize...

Only be sure always to call it please ‘research’.”

Although the dictionary definitions of plagiarism are clear enough, in practice their application is more difficult. All of us recognize the “bloody obvious” cases where a Shakespeare sonnet turns up in a high school poetry contest and certainly “cut-and-paste” and Internet search engines have facilitated such plagiarism. However, plagiarism software now makes this type of event much easier to detect. More difficult for science publishing is the reuse of descriptions of experimental methods; many experimental methods are essentially boilerplate and there are, after all, only so many ways to say that, “A was mixed with B to form C which was used in the next reaction”. Referencing the original source of the ‘methodology’ should, of course, be expected. Also difficult is “self-plagiarism” where portions of the author’s own work may be reused in a subsequent publication: how does one deal with the (fairly typical) process of incremental publication where work may appear as a meeting abstract, a brief or preliminary communication, a full paper and perhaps a final book chapter or review article, or even a second ‘review’ article that is minimally different from the original? At what stage of self-plagiarism should this be termed fraud, or is it fraud? The answer is clearly full and clear disclosure of the nature of the preceding publication(s) (and awareness of the nature of copyright law) and how the new publication derives from and expands on the original.

Plagiarism is, of course, not confined to the word of scientific publication. Kaavya Viswanathan, the Harvard student and currently celebrated author of the “chick-lit” novel, How Ophal Mehta got Kissed, got Wild, and got a Life, acknowledges that portions of the text were similar to work from another author (Smith 2006). And Vladimir Putin, President of Russia, was accused of incorporating into his PhD dissertation material from a previously published book by University of Pittsburgh professors David Cleland and William King (Allen-Mills 2006; Gaddy 2006).

For playwrights, artists and poets, at least in the past, it may have been considered acceptable to “improve” upon the works of their predecessors, but that is unlikely to pass without judgement today. But what about the authors of text books? As Richard Posner (2007) points out in his Little Book of Plagiarism, parts of T.S. Eliot’s widely acknowledged masterpiece The Waste Land (1922) are really “a tissue of quotations (without quotations marks).” Eliot seemingly acknowledges his and other’s faults when he states: “Immature poets imitate; mature poets steal; bad poets deface what they take, and good poets make it into something better, or at least something different.”

“Honestly, it wasn’t plagiarism—I had a dream”

We have all heard stories of plagiarism that have been vigorously denied by those accused, but do they have a genuine defence? Is there evidence for true “unintentional plagiarism”? Is it possible that the culprit has absorbed some fact or idea and honestly believed it was their invention? Cryptomnesia, or unconscious plagiarism, has been reported in several studies (see Brown and Murphy 1989; Marsh and Boyer 1993) thus providing credence to this potential defence against charges of scientific misconduct. According to Brown and Murphy prominent figures from Freud, Keller, and Nietzsche have all been accused of this version of plagiarism. And Kaavya Viswanathan also advanced this argument in defence of her plagiarism. The frequency of genuine cases of cryptomnesia remains unknown, but it is unlikely that it is at the pandemic levels that are needed to explain the reported high incidence amongst students.

What a good idea!

Less readily substantiated is the theft of ideas from grant applications and abuse of the peer review process so that the reviewer can benefit from the ideas of others whose grants the reviewer has just torpedoed. The US Office of Research Integrity has also taken this problem under its wing and it will be interesting to see the extent of the problem that emerges from their investigations (Ready 2006). Is legal recourse the approach? We would need to know who to sue when mistakes in the peer review process are made. Is it the deceitful reviewer, the committee chair or the funding organization? There can be quite subtle ways by which reviewers dismiss the work of others and, at the same time, promote the work that their team/institute is pursuing. Is this ethical? An example of unethical behaviour is promoting the use of stem cell technology for tissue repair over other approaches simply because stem cell technology is new and current and in vogue and a technique being used by the reviewer! We use this example NOT as an argument against stem cell technology as a promising area for research (which it clearly is), but rather an indication of a manipulative reviewer abusing the peer review process in order to promote their personal goals. Another frequently and often inappropriately used term to dismiss a grant is “descriptive”. Descriptive infers that the research is not “mechanism driven” and therefore will not provide the answer to the question being addressed. But what was the question being asked and has the reviewer really considered the hypothesis being tested? Research is progressive and with each advance it is anticipated that another piece of the overall puzzle is found and placed into the picture. If one takes the “reductionist” viewpoint then one needs to take this to the gene level, but then are not we missing what are happening at the organ, whole organism, and community level?

Academics and business—a marriage made in Heaven or in Hell?

There has, particularly during the past 10–20 years, been an increasing interest in promoting the business side of academic science with governments and universities helping to facilitate the licensing of intellectual property (IP) as well as the evolution of spin-off biotechnology companies. Indeed, there have been a number of spectacular successes, for instance, Chiron Corporation that was founded in 1981 by Professors Rutter, Penhoet, and Valenzuela: all three founders were professors of biochemistry in California (Dr Rutter was Chairman of the Department of Biochemistry and Biophysics, UCSF). The benefits of successfully spawning spin-off companies are multiple and, in particular, results in the creation of jobs for graduates as well as revenue for the inventors, for the institutions, as well as, by promoting employment and GDP, generating taxes for governments. There is also a societal impact in so far as it can be argued that such companies provide the impetus that facilitates the translation of a discovery to a beneficial product—be it a therapeutic, a device, or a new technology. Similarly licensing of IP can, in fact more rapidly than spin-offs, generate wealth.

But are licensing and the generation of spin-offs all roses? To start with, without doubt, there have been many more failures than successes when it comes to spin-offs. Secondly, most universities do not really have the in-house expertise to advice, nurture and manage the business interests of academics and to develop such expertise could prove to be an unprofitable drain on their budgets. Thirdly, most academics have minimal business aptitude and, furthermore, should they be developing and/or managing companies when they are employed as full-time academics? Fourthly, who really owns the IP? Universities have many, sometimes overly complex, models of IP ownership, but which model is best? This is often an emotional issue and, in many instances, the academics may also contribute personal funds to protect the IP. Fifthly, the situation of IP can become very tricky when trainees are involved, notably graduate students, and university administrators are advised to avoid this quagmire. For instance, who really “owns” the IP when it was “discovered” as part of a student’s thesis project? What recourse is there if part of a student’s thesis or publication is used as part of a patent application or license agreement without their knowledge or agreement? No doubt these problems arise, but how are they dealt with and, indeed, are they dealt with or just hushed up? Indeed, we suspect that many universities are ill equipped to handle such matters in a fair and transparent manner. Universities rarely take the high road and make the righteous decision in such investigations, partly in the hope of avoiding adverse publicity or fear of reprisal or litigation from the guilty party. Such decisions make it difficult for the whistle-blowers who themselves may feel threatened. Certainly, students should never be ‘employed’ to pursue a project when the question of IP and confidentiality is likely to impinge upon the advancement and goals of their academic program.

In any event, most universities do not make money from sale of their IP and some have argued that universities are, by over enthusiastically pursuing commercial objectives, losing sight of their principal objective, namely contributing to and maintaining the intellectual commons (inter alia Krimsky 2003; Leaf 2005; Triggle 2005a; Boettinger and Bennett 2006). Ultimately, universities may even lose their special role and be treated as just another commercial enterprise subject to the rules of the business marketplace. In Madey vs. Duke the Court of Appeals for the Federal Circuit disallowed a defence that the experimental (research) use of patented material without a license or royalty payments was appropriate in a university setting. The court revealed in the decision that, “.....Duke...like other research institutions..... is not shy in pursuing an aggressive patent licensing program from which it derives a not insubstantial revenue stream” (Eisenberg 2003; Madey vs Duke University 2002). After all when most universities are established from public funds they then have a competitive advantage over truly commercial companies.

Academic-industrial conflicts also arise when the industrial partner attempts to dictate the contents of, or even suppress, all or portions of a scientific paper (Triggle 2005a). The Journal of Occupational and Environmental Medicine has announced that it will retract a paper that it published in 1997 that claimed no association between cancer and hexavalent chromium (Zhang and Li 1997). This conclusion contradicted a previous paper by the same authors and in what the Wall Street Journal described as “a black eye for scientific publishing” it appears that the 1997 paper was “actually conceived, drafted and edited by consultants for the PG & E corporation” that was at the time involved in litigation over this very issue (Waldman 2005, 2006). Similarly, the New York Times has reported that articles published in journals that advocate one drug over another may actually reflect little more than a commercial message rather than a balanced and objective analysis (Carlat 2006). Thus, in a paper entitled “A review of the evidence for the efficacy and safety of trazodone in insomnia” (Mendelson 2005) it is noted that “Sepracor Inc assisted in the preparation of this manuscript and Dr Mendelson received compensation from Sepracor in support of the development of this manuscript”. The paper concluded with a cautionary comment about the safety of trazodone’s use, but what is not noted is that Sepracor is the manufacturer of Lunesta™ a non-generic insomniac agent that both competes with trazodone and is more expensive. Regardless of the validity of Mendelson’s conclusions these conflicts should be unacceptable in scientific publishing.

So, where are we?

First of all we do not wish to leave the reader with the impression that the peer review system is obscenely corrupt, that plagiarism is rife, that scientists are, by nature, attention-seeking frauds intent on being media stars and making a quick dollar—although some scientists have one or more or even all of these characteristics. Scientists are no different from any other groups in society and, like many other analogous comparisons, a few rotten apples will always be found. What scientists must ensure is that the public does not come to believe that is not just the apples but that the barrel itself is rotten. Indeed, when it comes to detecting fraud, new technologies should enable us to be much more analytical when it comes to peer review and issues of integrity and frauds are exposed, thus the process is evolving and the purpose of this review is, we hope, to speed up the evolutionary process—some may call this intelligent intervention. We also stress that we are certainly not the only scientists to argue for reform in the peer review system as well as the monitoring of science in general and the concerns that we have raised are shared by others whose contributions to this subject could not all be acknowledged in our brief overview (see also Horrobin 2001). What should now be apparent to the reader is that scientists, like other groups within society, consist of a majority of honest citizens together with a minority of less scrupulous individuals who, to varying degrees, will manipulate the system for their own benefit. Should this surprise us? The answer is “no”, but what needs to be changed is how such ‘manipulators’ are dealt with by society. We have already argued that investigations of suspected fraud should not only be pursued by the institution(s) where the alleged fraud was committed but should be referred to, as appropriate, national or international agencies (such as the equivalent of the ORI). Scientific fraud, in its many manifestations, is no different from any other form of fraud and should be dealt with as such with appropriate penalties. Accepting this philosophy will be a major step forward for science and for public confidence.

What should we do?

We can offer recommendations, but, given the potential that the publication of fraudulent data not only severely damages the credibility of scientific research but also affects the health of the population we believe that changes are urgently required. Furthermore, although we recognise that peer review depends on the good will and commitment of the reviewers, the peer review system needs to evolve into a process that is truly accountable and devoid of bias with any attempts by reviewers to sabotage, either through intent, dishonesty or incompetence, acted upon by appropriate, possibly legal, action. These changes require cooperation by both academic (and other) institutions and national and international bodies that have the power to investigate allegations of corruption.

Thus, we prefer to make just a limited number of specific recommendations that may set the stage for further discussion:

The establishment of the equivalent of the Office of Research Integrity (ORI) in other countries would be a major step forward for the recognition that scientific fraud is a serious and, potentially criminal matter. As already noted comparable offices have been established elsewhere (UK) and/or legislation is being considered (Norway).

Appropriate safeguards designed to protect both the whistleblower and the accused.

Processes whereby the apparent bias in peer review can be reduced are urgently required and should be evaluated. At the level ofjournal publications the “open” review process adopted by the journal Nature may be one such process and another potentially beneficial process would be the publication of “signed” précis of the reviews together with the published article. Transparency and accountability should always be prominent in peer review.

Heightened awareness and education at all levels concerning the seriousness of scientific fraud in all of its manifestations.

Scientists are not, of course, the only guilty people, and certainly not the guiltiest people. There is scarcely a day without a newspaper or television headline about some new criminal financial activity, where some public figure is not taking a “perp walk”, where some new abuse of human rights has not occurred, where a politician is found with a hand in the cookiejar or in bed with a secretary, or where the car sold to you as just driven by one old lady to go back and forth to church actually has 200 000 miles on the odometer and has been in three major wrecks. But we are not surprised by these events; indeed we almost anticipate them since public expectations of the ethics and honesty of businessmen, politicians and used car salesmen are not high. But the public expectation of science is much higher and thus the fall from grace is more significant and ultimately far more damaging.

The words of Francis Bacon (1561–1626) should be our guide:

“For myself, I found that I was fitted for nothing so well as for the study of Truth .... and as being a man that neither affects what is new nor admires what is old and that hates every kind of imposture.”

Other issues related to the safety of clinical trials and human testing have recently hit the headlines (Caplan 2006) and with globalization also being applied to clinical trials inevitably questions of ethical standards have arisen and stress the need for an international monitoring and, most likely, tightening of these standards (Jayaraman 2004; Nundy and Gulhati 2005). There is certainly the need for close scrutiny in this arena. The life-threatening problems that arose in March 2006 as a result of the phase 1 trial with TeGeniro’s “superagonist”, TGN 1412, of the immune system may lead to changes in the regulatory processes for the testing of new biologic drugs in humans (Sheridan 2006).

Finally, the truth is, of course, a sometime thing. The decision by Allen Dulles, then the Director of the Central Intelligence Agency, to place the words, “For ye shall know the truth and the truth shall make you free”, at the entrance to the CIA building seems now seems in light of past and ongoing events by the CIA to be a major irony. Science needs to hold to the original standards of John 8:32, and leave relative truth to the politicians, for that is their expertise.