Predicting Resistance Is (Not) Futile.

The treatment of many human diseases, from bacterial and viral infections to cancer, is becoming increasingly challenging as resistance to therapeutic agents continues to evolve. Identifying which drug(s) will effectively treat a patient is essential for achieving good patient outcomes; predictive methods that draw on knowledge of the chemical interactions between the compound and its target[1] have an important role to play.

In the past decade, next-generation sequencing has enabled large clinical data sets to be collected, allowing many of the genetic variations responsible for resistance to be inferred. Despite this, the majority of resistance-conferring mutations in cancers are seen with a low frequency or may even be unique to a patient.[2] Therefore, this is a problem which cannot be solved by collecting ever larger data sets: being able to predict the effect of individual protein mutations on the action of a drug will be essential for mutations with unknown resistance profiles.

Tyrosine Kinase Inhibitors (TKIs) are a class of first-line drugs used to treat chronic myelogenous leukemia. TKIs bind to the Abl kinase (Figure A), and resistance arises via missense mutations (Figure B) which usually confer resistance by decreasing how well the drug binds to its protein target. Hence, one route to resistance prediction is to calculate how the drug binding free energy changes upon introducing the genetic mutation into the target protein (Figure C).

Figure 1

(A) Tyrosine kinase inhibitors (TKI) bind the human Abl kinase. (B) Resistance to TKIs arise from missense mutations in Abl. (C) Effect of a mutation can be predicted by calculating how it affects the binding free energy of the TKI. (D) There are a range of methods to calculate how a protein mutation alters the binding free energy of a drug; here, we categorize them based on whether they are physics-based or data-driven.

Aldeghi et al.[1] applied three different methods to predict if a range of missense mutations conferred resistance to each of eight TKIs (Figure D). The first method belongs to the family of alchemical free energy methods which are derived from classical statistical mechanics. These methods were first applied to proteins over 30 years ago but, due to the large amounts of computational resource they require, have only recently found serious application thanks to the continued increase in computational speeds. By contrast, for their second method the authors take an entirely data-driven machine learning approach using randomized decision trees. Finally, the authors test Rosetta, a partly physics-based and partly empirical method based on an energy scoring function, to calculate binding free energies.[3]

This work builds on an earlier study by Hauser et al.[4] who found that a proprietary alchemical free energy method was able to predict the binding free energy of TKIs to the Abl kinase with reasonable accuracy. More importantly from a clinical perspective, the method was able to correctly categorize mutants as susceptible or resistant with 88% accuracy. Aldeghi et al.[1] show that a second alchemical free energy method built around open source computer code and molecular parameters classifies the mutations equally well, with a third method performing less well. The classification performance of Rosetta with the REF15 scoring function was comparable to the best results obtained by an alchemical free energy method. Machine learning performed least well of all three methods and was only successful when the algorithm was trained on a specific TKI/Abl kinase data set, exemplifying a lack of generalizability compared to the more physics-based Rosetta and alchemical free energy approaches. Interestingly, creating a consensus prediction by combining the results from alchemical free energy calculations using different parameters did not improve precision or recall, although in some cases combining the results of an alchemical free energy method and Rosetta did improve the predictions compared to either method on its own.

As the majority of missense mutations do not confer resistance and the more accurate techniques require a great deal more computer time, it will almost certainly be worthwhile to adopt a “funnelled” approach. Here, the effect of a genetic mutation would be first predicted using a machine learning model and then those with a moderate probability of conferring resistance could be examined in more detail using a physics-based method. While an alchemical free energy method might appear the most obvious choice here, other important considerations include the usability and reproducibility of the method.

Ultimately, some combination of these (and possibly other) methods could be used to inform treatment decisions. Since a binary paradigm of resistant/susceptible prevails clinically, the sensitivity and specificity of any predictive method is more important than its quantitative accuracy and precision. Where the balance should be drawn will depend on clinical factors: in bacterial infection the priority is avoiding false negatives (resistant infections incorrectly predicted as susceptible) not only to improve treatment success but also to prevent the spread of antimicrobial resistance. When fewer treatment options are available (as in many cancers), one may wish instead to minimize false positives.

Important work also remains to be done to ensure that the definition of “resistant” matches that used clinically. This is difficult for human cancers, and so here the authors assume that any mutation which leads to a 10-fold or greater drop in affinity confers resistance. The situation is more tractable in clinical microbiology where minimum inhibitory concentrations are defined by international bodies for pathogen/antibiotic combinations and these can be used to inform a predictive method.[5]

Computational chemistry has primarily focused on aiding lead identification and optimization by calculating the binding free energy for small molecules, and minor modifications thereof, to a protein target. This work[1] suggests that applying the same methods to predict the effect of protein modifications (mutations) on small molecule binding could be more straightforward and, ultimately, perhaps more useful as it may allow the design of drugs that could withstand the evolution of resistance for longer. Happily, Aldeghi et al. also demonstrate that alchemical free energy workflows constructed using free parameters and open source computer codes perform as well as proprietary ones, which is encouraging for future research and translation. Stepping back, this work is yet another example of a philosophical dilemma that will increasingly face chemists as larger experimental data sets become more prevalent: should we apply deductive methods based on an understanding of the underlying physics and chemistry or should we instead use machine learning approaches to make inferences?