QSP Versus the Rest: Let the Competition Commence!

The article by Stein and Looby challenges the community to benchmark quantitative systems pharmacology (QSP) models against simpler models. They describe examples from within and outside the field of pharmacology, which highlight the pitfalls of building complex models for prediction purposes. A key point raised is that QSP models may have poorer predictive performance than simpler models for a given task due to overfitting; therefore, guarding against this benchmarking should always be performed.

The article by Stein and Looby1 begins by describing why quantitative systems pharmacology (QSP) models have been gaining favor within the pharmaceutical industry. The key reason is the hope that by integrating more knowledge into a model we will be able to make accurate predictions of the outcomes of future experiments. This clearly has to be the goal of modeling endeavors within the pharmaceutical industry, increase predictive power, and, thus, productivity. There are practitioners of QSP who would argue that another goal, with a similar level of importance, is to learn about the biological system of interest. Therefore, if the model incorrectly predicts future experiments, this is not a worry as the model's structure can be adjusted to fit the new data. Thus, the QSP modeler would argue we have gained some new biological insight. However, to have any faith in the new model at some point we would hope that the model makes accurate predictions of future experiments. Therefore, even for learning about the biological system we would require that the model does make accurate predictions at some point, the sooner the better.

Moving on from the motivation behind QSP models, the perspective then describes the benefits of simplifying a complex model. It is important to note here that although a simpler model can be derived from the complex model through use of approximations, model lumping etc, this may not always be the best strategy. If the complex model contains a large degree of structural model error, which is likely to be the case for pharmacodynamics, then the simplified model may just end up amplifying the error. Thus, it may be more appropriate to ignore the complex model when building a simple model. This raises a question when wanting to benchmark QSP with simpler models. Should the QSP modeler be developing the simpler model, or vice versa, given that the mind‐set required for each task is different?

The article then moves on to discuss some of the limitations of QSP, which make accurate predictions challenging. In summary, the limitations can probably be reduced down to the following. How well do we truly understand the biology and how well can we measure it? Given there is a large degree of bias in experimental literature around positive results and the increased observation that many experimental data are not reproducible, can we really trust the “knowledge” that is in the literature and, thus, is it wise to integrate all this “knowledge”?

The article concludes by providing examples of where a degree of benchmarking models has been performed in certain areas of pharmacology, unsurprisingly there are not many before concluding that benchmarking should be done more often. The conclusion could be added upon by encouraging the development of prospective prediction competitions. Such competitions are gaining favor within the healthcare industry and could well provide a gauge in the difference in predictive power between QSP models and the alternatives (linear regression, simpler heuristic based model, etc) for a given prediction task.

In summary, the perspective article will hopefully encourage users/customers of QSP demand to see such models benchmarked against simpler models possibly through prospective prediction competitions. It is only through benchmarking will we truly see how useful QSP will be for pharmaceutical productivity. Let us move beyond eminence‐based modeling to evidence‐based modeling.

Conflict of Interest

The author declared no competing interests for this work.

Source of Funding

No funding was received for this work.