Best practices for repurposing studies.

All I can say is that on this earth there are pestilences and there are victims– and as far as possible one must refuse to be on the side of the pestilence.

—Albert Camus, La Peste [1].

As humanity is challenged with new pandemics and diseases, there is a desire to find treatments in a much faster time-frame than traditional discovery efforts can attain. One strategy has been the repurposing of compounds that have demonstrated efficacy and/or safety in human trials (see the articles in the special issue of Drug Discovery Today [2]) or through common usage. The goal is to identify compounds that may proceed directly to human efficacy studies. If a compound requires optimisation, then all its advantages are gone, and there is little to choose between it and any other hit structure, in terms of time to the clinic, the key metric. During the current Covid-19 pandemic, many papers have been written with the aim of identifying possible candidates for the repurposing strategy, but which unfortunately have been lacking in the proper controls or understanding to be accepted in this journal. This brief editorial tries to lay out some of the issues that authors should address in their studies. We do not hold a position for or against repurposing, but we must try to maintain certain scientific standards on behalf of our readership. We also have a duty not to mislead, resulting in studies that cost much human effort and divert resources from more promising areas, but to inform with genuine data.

Introduction

As humanity is challenged with new pandemics and diseases, there is a desire to find treatments in a much faster time-frame than traditional discovery efforts can attain. One strategy has been the repurposing of compounds that have demonstrated efficacy and/or safety in human trials (see the articles in the special issue of Drug Discovery Today [2]) or through common usage. The goal is to identify compounds that may proceed directly to human efficacy studies. If a compound requires optimisation, then all its advantages are gone, and there is little to choose between it and any other hit structure, in terms of time to the clinic, the key metric. Repurposing research should be based on experimental data. This should be introduced as soon as possible in the paper, so that the reader can distinguish this from in silico estimates. The absence of such data is a primary factor in rejecting a paper. There should also be a clear distinction in terms of the language used, between experimental data and calculated estimates.

The experimental data should also be significant: to achieve significance over non-specific events, the following rule-of-thumb should be applied. Experimental activity in a cell-based assay should be better than 1 microM, and in a biochemical assay, better than 100 nM. However, even results outside these ranges are of interest for approved drugs, as they may serve to eliminate these drugs from further consideration. Any critical evaluation of the pharmacopoeia is useful. Virtual screening results on their own are not sufficient.

What is safe?

Safety is a key concern, if one is to omit the usual animal studies that precede first dose in humans. Safety is a relative rather than an absolute characteristic. As Paracelsus puts it, ‘the dose makes the poison’. Drugs that are part of most repurposing databases (FDA list of approved drugs [3]) have been shown to be safe at the dosing protocols at which the original trials were conducted. Their ‘No Observed Adverse Effect Level’ (NOAEL, the highest experimental concentration that is without adverse effect in the organism) should be described, as should their pharmacokinetic (PK) profile (in terms of half-life and plasma levels). It should also be stated whether the drug is given chronically or acutely, orally or by injection, and at what dose. Curated datasets are starting to appear that address these issues for repurposing, as well as highlighting issues such as the current patent status [4]. These safety and PK considerations should be explicitly addressed in the discussion of any compound as a possible candidate. If a candidate has a NOAEL of 10 microM, and yet it needs to achieve levels of 50 microM for efficacy for the repurposed goal, that candidate should be eliminated. Similarly, the peak plasma concentration, Cmax, should also be known, allowing one to compute the concentration in plasma and estimate the concentration in a cell, if the target is intracellular. Pharmacodynamics (PD) also have a role to play. A drug with an unbound concentration of 50 nM will have no efficacy for a target against which its IC50 is 10 microM [5]. The same is true for solubility, which might also cap concentration levels to below those needed for the proposed levels of efficacy; for an example of how to review the medicinal chemistry of a compound, see here [6]. This should be a key discussion point for poorly soluble compounds, especially natural products. An understanding of NOAEL, solubility and Cmax can give some way to estimate the concentration of compound at the site of action, and hence the minimum efficacy or binding affinity that might be necessary. This will help to build confidence that the drug will show benefits when dosed according to established protocols. Application of generic rules for bioavailability (for example, rule-of-5) or computed ADMET properties [7] is not sufficient to address these questions, especially when the data is available.

On-target or off-target?

The first requirement of successful repurposing is that the effect should be target- or pathway-based, and not a non-specific effect controlled by physicochemical profile. The repurposing strategy is at its strongest when the proposed mode-of-action is very similar to the original purpose. The point of repurposing is to find a compound that is ready to go into humans directly, not one that can start off a discovery optimisation project. If we are trying to find optimisable hits, we can and should sample much larger spaces [8]. It cannot be assumed that analogues of a drug carry the same profile: if anything, the profile will have been optimised (whether by humans or by evolution) and even close analogues may not have the desired characteristics (and will not have been subject to the same clinical trials). If one is seeking a protease inhibitor, it is best to start with one [9]. This has been called hypothesis-driven repositioning [10].

Off target modes-of-action are problematic, in that one is hoping that a side-effect is more important than the main effect, and that levels of off-target efficacy can be reached before any NOAEL is breached. If there is no known equivalent action in the established pharmacology of a compound for the intended use, even greater care needs to be taken. Proper discussion of the link between effect on clinical symptoms of the disease and the proposed mode-of-action at the molecular level need to be carefully made. An example would be the proposed use of hydroxychloroquine as a treatment of Covid-19: without understanding how the supposed clinical observations translate into action at the molecular level, against a specified protein or pathway, one cannot use the observations to frame a sound and testable scientific hypothesis for docking studies. If a compound acts by boosting the immune response, there is not much sense in performing docking studies against viral proteins. Compounds that have appreciable activities and efficacies against a variety of targets (for example, some kinase inhibitors) might work by a mechanism related to their claimed mode-of-action [11], but careful studies are required to separate efficacious effects from harmful ones. This requires a careful understanding of whether one is trying to treat the symptoms of the disease [12], or the causative agent itself, the virus. Research has even indicated that the primary mechanism of action of many compounds with activity in cellular screens is due to phospholipidosis [13] (a phospholipid storage disorder that can be induced by cationic amphiphilic drugs [14], a side effect that is certainly not desirable). There are models for predicting this effect [15], but it would not be enough to rely solely upon such an assessment for repurposing.

Action on the target

It is here than an appreciation of the pharmacodynamics and pharmacokinetics comes into play. First, at what level does the drug inhibit the biological process efficaciously? Taking the example of mPro from the Cov-Sars-2 virus, should the enzyme be inhibited at 50, 90% or some other level? What does that equate to in terms of binding energy and then concentration? For how long must the level of blockade be maintained, to have a reasonable dosing regimen? If there is a PK requirement for a compound with nanomolar binding (efficacy is left to one side as affinity and efficacy are not necessarily related), we can estimate the minimum necessary binding score from studies that have examined the correlation between score and affinity for a variety of schemes and systems [16]. Usually scoring schemes overestimate experimental binding due to poor treatment of solvation, protein flexibility and other entropic factors. If a compound does not pass this test, it may be discarded. The next test is to compare the drug with a large sample of similar decoys [17]. If it not clearly distinguished in terms of computed score from the decoys, then it may be discarded. This is a very important control for identifying false positives; these often contain above-average numbers of h-bonding groups that are over-scored by enthalpically-driven scoring functions. It is also vital to distinguish between a measured binding affinity and a calculated score; many authors assume an absolute 1:1 correlation or at least a perfect ranking, and use the terms binding affinity and docking score interchangeably. We would point to work that shows this to be not the case (see the articles in these special issues [18, 19]). It seems one way to tell which score is being used, is to look at the number of significant figures being quoted!

Scoring methods

Docking is very useful as a filter for eliminating the large majority of structures that do not bind strongly to a target. It is not a good positive filter for identifying the best binders. Skilled modellers often trawl deep into the list of best scoring structures to find compounds worth further work [20]. Docking a small number of compounds and only retaining the top few based on the docking score alone is an exercise that is not valuable in terms of repurposing, as the controls are lacking [21]. Scores based on mm-gbsa are better but still do not have the desired accuracy [16]. Docking studies need to be compared to as many independent and validated controls as possible. Does the docking protocol reproduce known binding modes? Has a good decoy set been tried, to establish a baseline activity? Do the proposed compounds beat that baseline significantly? Are there experimentally determined values of affinity to compare to? Perhaps we can hope that docking will at least give us a reliable docking pose: sadly, for the case of mPro in SARS-CoV-2, this has been shown not to be the case [22], so even more care needs to be applied when using docking as a positive filter rather than a negative one.

If the recent history of the COVID-19 pandemic is used as an example, there have been several compounds that were proposed to have efficacy against the disease. Regardless of whether they were effective or not, that still leaves the question of the potential mode-of-action open. Docking cannot prove a mode of action unless all possible targets are examined. It is very tempting to try to show that the most currently promising treatment and the most promising target go together. As the pandemic has progressed, so hydroxychloroquine, remdesivir, azelastine, ivermectin and several other natural products have all come top of the docked lists in submitted manuscripts. This is confirmation bias; not every one of these compounds can be the one with highest affinity, as comparative and decoy docking would show, let alone more rigourous methods. The repurposing paradigm should show clear winners, not what compound is currently the focus of most attention.

Alchemical free energy perturbation approaches offer more hope, but these are expensive and not without their own issues [23]. Performing relative free energy calculations is more practical than absolute calculations, but for that one needs measured standards to be included in the analysis that are reasonably close in structure to the structures being predicted. There are now guides to best practice in this area [24]. Analysis of known drugs shows that the drug fall into small clusters of analogues, with strong inter-cluster diversity [4]. This is unfavourable to the application of free energy perturbation strategies to evaluate dissimilar chemotypes. In summary, the best path forward is to measure the affinity of the proposed compounds using best practices [25], and then make a comparison to the requirements of PK/PD for the target. In this way one can determine better the mode of action, and relate this to any phenotypic observations.

Conclusions

The point of repurposing is to find a compound that is ready to go into humans directly, not one that can start off a discovery optimisation project. During the year 2020, the number of submissions to this journal increased by 50%, and much of the increase came from highly flawed repurposing studies. The time and expertise of our reviewers are greatly valued by the editors, especially in a time of increased demand; we need to use this resource responsibly. Our role is to ensure that manuscripts reach a minimal standard at which we can ask reviewers to give up their time.

Authors writing repurposing studies are advised to take into account the factors described above and summarized in Table 1, especially around the use of computed in silico scores as surrogates for experimental data. While the factors in Table 1 do not form a rigid checklist, addressing them explicitly and with care will facilitate the jobs of the editors, and will increase the possibility of review and acceptance.

Table 1

Factors to be considered before submission of a repurposing study

Is there experimental evidence to support the proposed mode of action and to validate any calculated data?	Present this experimental data explicitly Distinguish carefully between measured and calculated data
	Be careful to check cited references, to ensure they do contain experimental data and not in silico estimates
	Is the mode of action specifically (target or pathway) or non-specifically (physicochemical property) driven?
Are the experimental activities significant?	Cell-based assay: better than 1 microM
	Biochemical assay, better than 100 nM
	Virtual screening results on their own are not sufficient
Is there a clear causal link between the proposed mode of action and any clinical observations?
Can be it shown that the candidate should be safe and efficacious at established dosing protocols?	Compare the concentrations required for effect at the target with the NOAEL and PK parameters