Fernando Alarid-Escudero1, Richard F MacLehose2, Yadira Peralta3, Karen M Kuntz1, Eva A Enns1. 1. Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, MN. 2. Division of Epidemiology and Community Health, University of Minnesota School of Public Health, Minneapolis, MN. 3. Department of Educational Psychology, University of Minnesota, Minneapolis, MN.
Abstract
BACKGROUND: Calibration is the process of estimating parameters of a mathematical model by matching model outputs to calibration targets. In the presence of nonidentifiability, multiple parameter sets solve the calibration problem, which may have important implications for decision making. We evaluate the implications of nonidentifiability on the optimal strategy and provide methods to check for nonidentifiability. METHODS: We illustrate nonidentifiability by calibrating a 3-state Markov model of cancer relative survival (RS). We performed 2 different calibration exercises: 1) only including RS as a calibration target and 2) adding the ratio between the 2 nondeath states over time as an additional target. We used the Nelder-Mead (NM) algorithm to identify parameter sets that best matched the calibration targets. We used collinearity and likelihood profile analyses to check for nonidentifiability. We then estimated the benefit of a hypothetical treatment in terms of life expectancy gains using different, but equally good-fitting, parameter sets. We also applied collinearity analysis to a realistic model of the natural history of colorectal cancer. RESULTS: When only RS is used as the calibration target, 2 different parameter sets yield similar maximum likelihood values. The high collinearity index and the bimodal likelihood profile on both parameters demonstrated the presence of nonidentifiability. These different, equally good-fitting parameter sets produce different estimates of the treatment effectiveness (0.67 v. 0.31 years), which could influence the optimal decision. By incorporating the additional target, the model becomes identifiable with a collinearity index of 3.5 and a unimodal likelihood profile. CONCLUSIONS: In the presence of nonidentifiability, equally likely parameter estimates might yield different conclusions. Checking for the existence of nonidentifiability and its implications should be incorporated into standard model calibration procedures.
BACKGROUND: Calibration is the process of estimating parameters of a mathematical model by matching model outputs to calibration targets. In the presence of nonidentifiability, multiple parameter sets solve the calibration problem, which may have important implications for decision making. We evaluate the implications of nonidentifiability on the optimal strategy and provide methods to check for nonidentifiability. METHODS: We illustrate nonidentifiability by calibrating a 3-state Markov model of cancerrelative survival (RS). We performed 2 different calibration exercises: 1) only including RS as a calibration target and 2) adding the ratio between the 2 nondeath states over time as an additional target. We used the Nelder-Mead (NM) algorithm to identify parameter sets that best matched the calibration targets. We used collinearity and likelihood profile analyses to check for nonidentifiability. We then estimated the benefit of a hypothetical treatment in terms of life expectancy gains using different, but equally good-fitting, parameter sets. We also applied collinearity analysis to a realistic model of the natural history of colorectal cancer. RESULTS: When only RS is used as the calibration target, 2 different parameter sets yield similar maximum likelihood values. The high collinearity index and the bimodal likelihood profile on both parameters demonstrated the presence of nonidentifiability. These different, equally good-fitting parameter sets produce different estimates of the treatment effectiveness (0.67 v. 0.31 years), which could influence the optimal decision. By incorporating the additional target, the model becomes identifiable with a collinearity index of 3.5 and a unimodal likelihood profile. CONCLUSIONS: In the presence of nonidentifiability, equally likely parameter estimates might yield different conclusions. Checking for the existence of nonidentifiability and its implications should be incorporated into standard model calibration procedures.
Authors: Douglas C A Taylor; Vivek Pawar; Denise T Kruzikas; Kristen E Gilmore; Myrlene Sanon; Milton C Weinstein Journal: Pharmacoeconomics Date: 2012-02-01 Impact factor: 4.981
Authors: David M Eddy; William Hollingworth; J Jaime Caro; Joel Tsevat; Kathryn M McDonald; John B Wong Journal: Med Decis Making Date: 2012 Sep-Oct Impact factor: 2.583
Authors: Rebecca M Turner; David J Spiegelhalter; Gordon C S Smith; Simon G Thompson Journal: J R Stat Soc Ser A Stat Soc Date: 2009-01 Impact factor: 2.483
Authors: Fernando Alarid-Escudero; Amy B Knudsen; Jonathan Ozik; Nicholson Collier; Karen M Kuntz Journal: Front Physiol Date: 2022-05-09 Impact factor: 4.755
Authors: Anna Heath; Natalia Kunst; Christopher Jackson; Mark Strong; Fernando Alarid-Escudero; Jeremy D Goldhaber-Fiebert; Gianluca Baio; Nicolas A Menzies; Hawre Jalal Journal: Med Decis Making Date: 2020-04-16 Impact factor: 2.583
Authors: Fernando Alarid-Escudero; Eline M Krijkamp; Petros Pechlivanoglou; Hawre Jalal; Szu-Yu Zoe Kao; Alan Yang; Eva A Enns Journal: Pharmacoeconomics Date: 2019-11 Impact factor: 4.981