A new nonlinear mixture response surface paradigm for the study of synergism: a three drug example.

A flexible approach to response surface modeling for the study of the joint action of three active anticancer agents is used to model a complex pattern of synergism, additivity and antagonism in an in vitro cell growth assay. The method for determining a useful nonlinear response surface model depends upon a series of steps using appropriate scaling of drug concentrations and effects, raw data modeling, and hierarchical parameter modeling. The method is applied to a very large in vitro study of the combined effect of Trimetrexate (TMQ), LY309887 (LY), and Tomudex (TDX) on inhibition of cancer cell growth. The base model employed for modeling dose-response effect is the four parameter Hill equation [1]. In the hierarchical aspect of the final model, the base Hill model is treated as a function of the total amount of the three drug mixture and the Hill parameters, background B, dose for 50% effect D50, and slope m, are understood as functions of the three drug fractions. The parameters are modeled using the canonical mixture polynomials from the mixture experiment methodologies introduced by Scheff [2]. We label the model generated a Nonlinear Mixture Amount model with control observations, or zero amounts, an "NLMAZ" model. This modeling paradigm provides for the first time an effective statistical approach to modeling complex patterns of local synergism, additivity, and antagonism in the same data set, the possibility of including additional experimental components beyond those in the mixture, and the capability of modeling three or more drugs.