The Ecstasy and Agony of Assay Interference Compounds.

The ecstasy of discovering a new hit from screening can lead to a highly productive research effort to discover new bioactive compounds. However, in too many cases this ecstasy is followed by the agony of realizing that the compounds are not active against the desired target. Many of these false hits are Pan Assay INterference compoundS (PAINS)[1] or colloidal aggregators.[2] Whether the screen is conducted in silico or in the laboratory and whether screening libraries, natural products, or drugs are used, all discovery efforts that rely on some form of screening to identify bioactivity are susceptible to this phenomenon. Studies that omit critical controls against experimental artifacts caused by PAINS may waste years of research effort as useless compounds are progressed.[3−8] The American Chemical Society (ACS) is eager to alert the scientific community to this problem and to recommend protocols that will eliminate the publication of research articles based on compounds with artificial activity. This editorial aims to summarize relevant concepts and to set the framework by which relevant ACS journals will address this issue going forward.

Alarmingly, up to 80–100% of initial hits from screening can be artifacts if appropriate control experiments are not employed. The source of this artificial behavior has been thoroughly summarized in the literature.[9−12] Misleading assay results can arise through a variety of mechanisms including covalent protein reactivity,[13] redox activity, interference with assay spectroscopy,[14−16] membrane disruption,[17] decomposition in buffers,[18] and the formation of colloidal aggregates.[2,19,20] If not properly controlled, colloidal aggregation is perhaps the most common artifact from high-throughput screening: between 1 and 3% of molecules in many screening libraries will aggregate at relevant concentrations and up to 95% of “hits” identified from a screen can be assigned as aggregates,[21] and the colloids that they form inhibit,[20−22] or occasionally activate, proteins.[23] PAINS molecules can be synthetic in origin or derived from natural products; the latter have been termed Invalid Metabolic PanaceaS, or IMPS.[24] Even marketed drugs can aggregate and may also contain PAINS chemotypes. Over 60 FDA-approved and worldwide drugs contain PAINS chemotypes,[25] and about the same number have been shown to aggregate.[26] Although some drugs can contain PAINS and can aggregate at micromolar concentrations, such examples do not imply that any molecule that acts via a PAINS or aggregation mechanism can become a drug. Hence, noting or “flagging” any PAINS-containing hits and performing detailed follow-up experiments are essential to validate that the function of the molecule is as expected prior to discarding it from further consideration.[27] However, it is important to realize that no PAINS-containing drug has ever been developed starting from a protein-reactive PAINS target-based screening hit.[28]

Publicly available filters can help to identify PAINS and aggregators (e.g., http://zinc15.docking.org/patterns/home, http://www.cbligand.org/PAINS/, http://fafdrugs3.mti.univ-paris-diderot.fr/, http://advisor.docking.org), but these tools will not comprehensively identify all compounds with PAINS-like or colloidal behavior, and they may also inappropriately label a compound as an artifact when it is not.[29,30] Any in silico filter should therefore be augmented by experimental follow-up, a detailed practical guide for which has recently been published.[31] Such validation experiments include classic dose response curves, lack of incubation effects, imperviousness to mild reductants, and specificity versus counter-screening targets. If a molecule is flagged as a potential PAINS or aggregator using published patterns but is well-behaved by these criteria, it may be a true, well-behaved ligand. Ultimately, genuine SAR combined with careful mechanistic study provides the most convincing evidence for a specific interaction.[30,32] Covalent and spectroscopic interference molecules act via specific physical mechanisms, for which controls are known (see section ). Colloidal aggregation, fortunately, is readily identified by rapid mechanistic tests and by counter-screening (see section ).

While this editorial focuses on target-based screening, the issue of PAINS is also relevant to phenotypic screening and to drug repurposing studies, and it is obvious that rational interpretation and optimization of cellular activity with an inherently reactive chemotype may be difficult if not impossible.[33] Further, membrane perturbation becomes an additional promiscuity mechanism[17] and is very likely a contributing reason for the prevalence of IMPS in scientific databases and literature.[24] Whether PAINS and/or IMPS motifs are present, the common requirement of comprehensive and logical SAR is of paramount importance for any phenotypic screening hit, and optimization to well under micromolar levels of activity should be demonstrated.

Controls for Artifactual Assay Activity

Irreversible Inhibitors

Unless one is specifically screening for selective covalent modifiers, irreversible inhibitors—either acting themselves through a reactive center or representing the activity of an impurity—are typically undesired artifacts. A rapid counter-screen for irreversible inhibition is to incubate the target protein at 5× its normal assay concentration and the hit at 5× its apparent IC50, and after incubation, dilute them 10-fold (other IC50 ratios may of course be chosen). If inhibition is rapidly reversible, the inhibition on dilution should drop to about 33% of full inhibition on dilution (about 40% of the value at 5× the IC50). If dilution changes the inhibition little, it supports covalent activity. Legitimate slow off-rate inhibition is another alternative, but such molecules are rare among initial screening hits. This experiment will only work for soluble proteins, but related experiments to measure off-rate may be adapted for membrane proteins. More generally, a time-dependent increase in apparent inhibitory potency suggests irreversible binding. Lack of dissociation determined by direct measurement of ligand kinetics using biophysical methods such as surface plasmon resonance also demonstrates irreversibility. Inhibitors with electrophiles need not react with target proteins to be problematic. Their reversible reactions with cellular thiolates, such as glutathione, can render them inactive in cells.[99] It may be necessary to use several techniques to differentiate between covalent/nonreversible, covalent/reversible, and pseudoirreversible inhibitors.

PAINS Molecules

The chemotypes represented by these molecules often occur among promiscuous molecules that fail to progress. Most PAINS are dominated by a few chemotypes that are readily recognized.[29] Several in silico tools are available to identify these groups, including at http://www.cbligand.org/PAINS/, http://zinc15.docking.org/patterns/home, and http://fafdrugs3.mti.univ-paris-diderot.fr/. PAINS molecules act through several interference mechanisms, including all those described herein, and there is no single diagnostic test for the entire suite of bad actors. We recommend counter-screening the molecule against unrelated targets, as well as determining whether it competes with a ligand known to bind to the site and whether its concentration–response curves are well-behaved (e.g., has a Hill coefficient close to 1, or a strong mechanistic reason to differ from 1). PAINS frequently make it through to peer-reviewed publications, as protein reactivity can be subtle and selectivity over counter-screens may be exhibited. Therefore, thorough and logical SAR is the most important criterion that distinguishes a PAIN from a non-PAIN. As for any screening hit, literature review for evidence of chemotype promiscuity is essential, and in this context, Badapple[34] is an excellent resource that merits special mention.

Spectroscopic Interference Compounds and Compounds That Inhibit Reporter Enzymes

Compounds that absorb light or fluoresce in a region used to measure activity, or compounds that inhibit a reporter enzyme, like luciferase,[12,14−16] can appear to be active, but in fact are simply interfering with the assay. Spectroscopic interference should change linearly with concentration, following Beer’s law, rather than log-linearly as in a single site isotherm. Inhibitors of the reporter enzyme require a counter-screen. For all assay detection methods, it is critical to determine the propensity of screening hits to interfere with the detection signal by running an artifact or interference assay measuring the effect of the compound on the signal detection reagents.

Colloidal Aggregation

Perhaps the largest single source of artifacts in early discovery is colloidal aggregation by small molecules.[26] These particles, typically between 50 and 1000 nm in radius, nonspecifically adsorb protein, partially denaturing them. About two percent of molecules in a typical screening deck will aggregate at relevant concentrations, ensuring that hits reflecting colloid formation dominate in screens, both virtual and empirical, which do not control for them. Fortunately, molecules that act as aggregators can sometimes be recognized computationally (http://advisor.docking.org),[2,21,22] and better still, this mechanism may be readily controlled experimentally:

If activity can be attenuated by small amounts of nonionic detergent, the compound is likely an aggregator. A typical protocol involves 0.01% v/v freshly prepared Triton X-100 or 0.025% v/v Tween-80[35] for membrane or cell assays.

Direct observation of particles in the 50 to 1000 nm size range by dynamic light scattering (DLS). Formation of particles does not guarantee promiscuous inhibition, but it is a worrying sign.

For cell-based assays, colloidal particles can be precipitated by centrifugation of the medium before the assay is run. If the compound is much more effective before spin-down, it suggests colloidal aggregation. As an aside, it is critical to demonstrate that the compound is active at concentrations substantially lower than those producing cellular toxicity to show that the apparent activity is not simply due to cell death. In activity assays in which cytotoxicity is the desired end-point (e.g., anticancer assays), the compound should show high selectivity for cancer over normal cells.

Noncompetitive inhibition with high Hill slopes. There are classical reasons for noncompetitive inhibition and for cooperative binding, but the latter is rare in early discovery and the two together suggest aggregation.

Attenuation of inhibition by increasing target concentration. Except when the receptor concentration to Ki ratio is high,[36−38] increasing receptor concentration should not affect inhibition for well-behaved inhibitors. However, inhibitory activity will be much reduced for colloidal aggregators, and an increase in the steepness of the response curve will be observed. This experiment can only be used for soluble proteins.

Potential aggregators can be counter-screened for inhibition of enzymes like AmpC β-lactamase, trypsin, or malate dehydrogenase, which are highly sensitive to compound aggregation.[39] These enzymes are convenient because perturbations like detergent addition, which are not well-tolerated by some systems, are readily tolerated by these enzymes.

Regardless of whether or not one suspects that a molecule is a bad actor, detailed biophysical testing of new inhibitors for mechanism is always useful and can accelerate a drug discovery campaign. There is an understandable tendency to fall in love with early hits, but hard experience[40] shows that early hits can be fool’s gold and distract from more promising molecules that emerge later. Measuring and publishing full concentration–response curves is a simple but crucial way to retain focus on only the most interesting molecules; much can be learned from the steepness of the curve and how well it is sampled.[41] A step further is to measure the full binding constant, either through determination of the Ki by kinetic analysis, by radioligand displacement, or by reporter-free methods such as isothermal titration calorimetry, surface plasmon resonance, or related techniques. Here too, full curves should be measured and reported.

In light of these concerns, the participating ACS journals plan to uphold the standards above to ensure that all compounds for which activity is reported demonstrate activity commensurate with expectations (i.e., the compound is binding to the expected pocket and accompanied by thorough SAR). Active compounds from any source must be examined for known classes of assay interference compounds, and this analysis must be provided in the general experimental section. For compounds with potential assay interference liability, firm experimental evidence must be presented from at least two different assays, both of which report that the compounds are specifically active and that the apparent activity is not an artifact. Other issues that need to be considered in this context are the purity of the compound, stability in assay buffers, cysteamine or glutathione (GSH) reactivity, and a review of the literature for previous activities reported for the compound or compound class. The goal here is not to eliminate a priori all compounds that may resemble PAINS or colloidal aggregators—this cannot always be done to 100% confidence, and even molecules that appear to have progressable SAR can still be artifacts.[32] Rather, the goal is to ensure that compounds with an inbuilt tendency toward this behavior are well-vetted before publication, or indeed before submission for publication. This will diminish the number of articles that mislead the field. These new standards will bring clarity to medicinal chemistry and chemical biology and further ensure the already high level of science published in ACS journals.