Winnowing the chaff of charlatanism from the wheat of science.

Experts and lay people alike can sometimes find it difficult to demarcate the absurd. Here I propose a set of criteria that may be helpful in achieving this in the realm of healthcare: falsifiability, plausibility and some hallmarks of pseudoscience. Applying this method is unlikely to be fool-proof but it might be a valuable aid in discriminating credible from incredible health claims.

‘The scientific theory I like best is that the rings of Saturn are composed entirely of lost airline luggage’

Mark Russell

In medicine, we are frequently exposed to claims that fly in the face of science, perhaps nowhere more than in the area of complementary/alternative medicine (CAM), e.g. Both lay people and healthcare are sometimes unable to winnow the chaff of charlatanism from the wheat of science (1). This is not merely a nuisance but can generate serious harm, e.g. by ‘furnishing an escape from science and a line of retreat back to the old primitive beliefs of the cave man’ (2). Scientists might think they are able to tell reason from unreason. But homeopaths, chiropractors, iridologists, etc. believe that too. Who is right and who is wrong? A reliable method for demarcating the absurd (1) is called for.

The higher the dilution of a remedy the more powerful are its effects (homeopathy).

All human illnesses originate from ‘malalignments’ of the spine and can be treated with spinal adjustments (chiropractic).

Abnormalities of the color pattern of the iris provide diagnostic clues for internal diseases (iridology).

Human diseases are due to an imbalance of the two life forces yin and yang (acupuncture).

The function of inner organs can be influenced by manipulating specific zones located on the sole of the foot (reflexology).

Popper's principle that a hypothesis must be falsifiable can often be most helpful in this pursuit. The perhaps commonest defense of absurd claims is that science is not an appropriate tool for testing them. This is clearly untrue in the realm of CAM where all claims are testable; certainly the ones listed above are falsifiable in one way or another. Thus lack of falsifiability is not always a specific indicator of absurdity. Proponents of Absurd Claims (PACs) might argue that their claims are currently not sufficiently tested; concluding that they are wrong is strictly speaking impossible in such a situation. The more irrational a claim is, the smaller the likelihood that anyone will test it. Therefore many of the most absurd claims remain insufficiently tested. As a consequence, PACs are often eager to remind us that the absence of evidence is not evidence of absence. The point I am trying to make is simple: we need more tools for discrimination. My suggestion is to apply two additional criteria, in addition to Popper's falsifiability: plausibility and pseudoscience.

Plausibility relates to the question whether the claim and its underlying assumptions are in agreement with the known facts. If, for instance, homeopaths tell us that less is more or that nothing is something, we have little difficulties in showing that this is not supported by the known laws of nature. Homeopaths might counter that there are theories and experimental data that could explain the ‘memory of water’. The test of plausibility may therefore not always suffice. It relies on an ever-changing knowledge-base; what seems implausible today could become plausible tomorrow. To accurately identify absurd claims, we might need an additional tool.

The second criterion, I suggest, is the degree to which any claim carries the hallmarks of pseudoscience. There are numerous characteristics of pseudoscience, and different authors have suggested different sets of criteria (1). In the realm of CAM, four relatively simple characteristics have, in my experience, proved to be helpful.

Intolerance: Many PACs are consumed with evangelic zeal and find it hard to accept or even consider well-reasoned criticism or debate. Anyone who has tried to have a rational discussion with someone making irrational claims will have experienced this phenomenon. As intolerance can exist everywhere, its discriminating power is, of course, low and further criteria are required.

Selectivity: Most PACs tend to ignore facts that contradict their own assumptions. Instead they favor selected anomalous data or anecdotal findings which apparently support their notions. Clinical trials, for instance, are designed to overcome the many biases associated with simple observation. Whenever their results fail to confirm their belief, PACs insist that, for this or that reason, case reports, observational studies or years of experience are preferable. In arguing their case, PACs often seem to first formulate their conclusions, then selectively identify those bits of information that apparently confirm them.

Paranoia: Many PACs believe in conspiracy theories which posit that ‘the establishment’ is determined to suppress their views or findings. The world wide web, for instance, is full with suggestions that ‘big pharma’ is conducting a campaign against ‘alternative cancer cures’ such as laetrile or shark cartilage. Anyone who points out what the evidence really shows is likely to be accused of being part of the conspiracy.

Misuse: Some PACs misuse science, for instance, by using terminology like energy, chaos theory, quantum mechanics or entanglement in inappropriate contexts, devoid of their actual meanings (3). Accepted standards are rejected and double standards are proposed for their own area. In case this strategy fails, other means might be employed, including outright fraud (4).

Practical application: How can we use these criteria in practice so that we become less vulnerable to the claims of absurdity? Recent history shows that we still spend millions for testing hypothesis that were absurd in the first place (5). This money could be put to better use if we managed the demarcation of the absurd more effectively (1).

When confronted with a ‘suspicious claim’, my advice is to first check whether it is testable and whether it has already been tested. What, for instance, is the evidence that highly dilute homeopathic remedies can affect our health? A straight forward Pubmed search will produce plenty of data. Unfortunately the evidence is likely to be complex, confusing and contradictory. Whenever this first step does not generate a clear answer, identify the assumptions behind the claim and ask whether they are plausible. Is it really reasonable to assume that diluting a medicine can make it more potent, as homeopaths assume? In case the answer is still not certain, look out for the hallmarks of pseudoscience. This might involve some sort of exchange with the PACs who promote the claim in question. Do they argue rationally? Do they ignore evidence that does not fit their assumption? Do they employ scientific terms correctly?

This step-wise approach is unlikely to be fool-proof but it will, I hope, increase our chances to winnow the chaff of charlatanism from the wheat of science (1).